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ABSTRACT 


Defense  planners  must  strive  to  develop  and  incorporate  new,  efficient  procedures 
to  allocate  scarce  resources  in  varied  complex  environments.  We  consider  two  discrete¬ 
time,  discrete-space  search  effort  allocation  situations.  Both  involve  the  employment  of 
an  imperfect  sensor,  which  is  subject  to  both  false-positive  and  false-negative  errors.  The 
area  of  interest,  comprised  of  several  disjoint  area-cells,  contains  a  single  target  of 
interest.  In  the  first  situation,  the  target  moves  according  to  a  Markovian  transition 
matrix,  which  is  unknown  to  the  sensor  operator.  The  objective  is  to  estimate  the  target’s 
steady-state  distribution,  using  only  the  sensor’s  detection  signals  and  knowledge  of  its 
false-positive  and  false-negative  rates.  The  second  situation  considers  a  stationary  target, 
wherein  the  objective  is  to  detennine  the  area-cell  occupied  by  the  target,  in  the  fewest 
expected  number  of  investigations,  to  within  certain  operator-prescribed  error  tolerances. 
We  develop  an  adaptive  algorithm  based  on  stochastic  approximation  for  the  first 
situation,  and  show  that  the  resultant  rate  of  error  in  detennining  target  presence/absence 
in  any  area-cell  converges  to  zero  at  the  fastest  possible  rate.  We  propose  a  sequential 
elimination  procedure  for  the  second  situation,  which  provides  an  efficient  determination 
of  target  location  and  guarantees  its  error  rate  not  to  exceed  the  operator-prescribed 
tolerance. 
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EXECUTIVE  SUMMARY 


Today’s  operational  planners  and  sensor  operators  are  challenged  with  scarce 
resources  in  terms  of  both  time  and  sensor  assets.  One  may  argue  that  recent 
advancements  in  technology  have  slowed  the  growth  of  innovative  employment  of 
tactics,  techniques,  and  procedures  (TTPs),  making  sensor  operators  increasingly  more 
dependent  upon  that  technology.  Defense  planners  must  strive  to  develop  and 
incorporate  new,  efficient  procedures  to  allocate  scarce  resources  in  many  different 
complex  environments.  Any  efficiency  that  can  be  gained,  however  small,  may  have  a 
compound  effect  over  time  on  overall  combat  readiness,  by  freeing  up  precious  assets  to 
perform  other  time-sensitive,  critical  sensing  actions. 

In  this  thesis,  we  consider  two  particular  search  effort  allocation  situations  of 
operational  interest.  We  begin  by  describing  some  characteristics  common  to  both 
situations,  then  follow  with  a  discussion  of  both  situations’  unique  properties  and  a 
discussion  of  our  proposed  models.  Both  situations  allow  us  to  contend  with  the  fact  that 
search  sensors  are  imperfect;  i.e.,  they  are  subject  to  declaring  a  target  present  when  it  is 
in  fact  absent  (false-positive  error),  as  well  as  to  declaring  a  target  absent  when  it  is  in 
fact  present  (false-negative  error).  The  Area  of  Operational  Interest  (AOI)  for  both 
situations  is  comprised  of  a  grid  of  discrete,  non-overlapping  area-cells,  each  cell  having 
its  own  associated  values  for  sensor  error  rates.  These  area-cells  might  be  defined  by 
geo-political  borders,  terrain  features,  or  some  arbitrary  grid  system  of  tactical 
significance  to  the  operator,  and  need  not  be  uniform  in  size  nor  shape.  Both  situations 
involve  a  single  target  of  interest  (TOI),  located  somewhere  within  the  AOI.  The  TOI  is 
unintelligent,  in  the  sense  that  it  does  not  react  to  any  sensing  action.  We  treat  time  in 
both  of  these  situations  in  terms  of  discrete  time-steps.  The  operator  makes  one 
investigation  into  one  area-cell  per  time-step.  Both  situations  deal  with  the  allocation  of 
search  sensors,  which  implies  that  we  are  concerned  with  the  placement  of  sensors,  and 
that  those  sensors  are  not  restricted  to  follow  any  particular  path.  In  contrast,  a  search 
path  problem  might  impose  such  a  restriction;  say,  for  example,  only  immediately 
adjacent  area-cells  may  be  investigated  on  subsequent  time-steps.  It  may  be  assumed 
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either  that  sufficient  time  exists  between  investigations  so  that  a  single  sensor  may  be  re¬ 
positioned  to  any  other  area-cell,  or  that  each  area-cell  contains  one  pre-positioned 
sensor,  but  the  operator  may  process  only  one  of  those  sensors  in  any  time-step. 

The  first  situation  deals  with  a  randomly  moving  target,  whose  underlying  pattern 
of  motion  results  in  some  steady-state  distribution  over  time.  This  suggests  that  this 
particular  circumstance  is  concerned  with  long-term  Intelligence,  Surveillance  and 
Reconnaissance  (ISR)  operations,  where  the  number  of  available  search  opportunities  for 
the  operator  is  rather  substantial.  For  a  simplified  illustration  of  this  steady-state  concept 
in  a  two-cell  AOI,  this  may  mean  that,  in  the  long-run  average,  the  TOI  is  present  in  area¬ 
cell  “jc”  25%  of  the  time,  and  in  area-cell  “y”  75%  of  the  time.  Of  course,  the  underlying 
true  steady-state  distribution  of  the  TOI’s  location  is  ultimately  unknown  to  the  sensor 
operator,  who  must  use  only  the  (imperfect)  detection  signals  provided  by  the  sensors, 
along  with  knowledge  of  their  associated  error  rates,  to  detennine  an  estimate  of  this 
steady-state  distribution. 

The  model  developed  in  this  thesis  to  deal  with  this  first  situation  is  an  adaptive 
model,  meaning  that  it  provides  a  dynamic  allocation  plan  based  on  new  information  as  it 
becomes  available.  We  show  that  our  particular  procedure  converges  to  the  true  steady- 
state  distribution  for  a  large  number  of  search  opportunities,  and  that  the  error  rate  for 
determining  target  presence/absence  in  any  cell  converges  to  zero  more  quickly  than  with 
other  allocation  schemes.  The  result  implies  a  cost-savings  to  the  sensor  operator  and  the 
operational  planner,  allowing  precious  assets  to  be  freed  up  to  perform  other,  time-critical 
sensing  evolutions. 

In  the  second  search  allocation  situation  of  this  thesis,  we  are  concerned  with  a 
stationary  target  hidden  somewhere  in  the  AOI.  Likely  candidates  fitting  this  template 
might  include  an  insurgent  in  hiding,  an  Improvised  Explosive  Device  (IED),  or  a 
downed  friendly  aircraft  for  search  and  rescue.  The  objective  in  this  circumstance  is  to 
determine,  in  the  smallest  expected  number  of  investigations,  the  area-cell  in  which  the 
target  is  located.  Of  course,  since  the  sensors  are  imperfect,  there  is  no  guarantee  that  the 
answer  is  correct,  so  in  this  case  the  answer  must  be  framed  with  some  sort  of  confidence. 
To  accomplish  this,  the  operator  prescribes  an  error  tolerance.  For  example,  an  error 


tolerance  of  5%  would  mean  that  the  operator  is  willing  to  accept  that  the  model  provides 
a  correct  determination  of  target  presence  at  least  95%  of  the  time.  Naturally,  the  larger 
the  error  tolerance  the  operator  is  willing  to  accept,  the  more  quickly  the  operator  can 
expect  to  make  a  determination.  Conversely,  a  small  tolerance  for  error  could  mean 
many  more  search  attempts  expended  to  make  a  determination. 

To  handle  this  second  situation,  we  develop  a  family  of  sequential  elimination 
procedures.  These  procedures  work  in  stages;  during  each  stage,  all  possible  area-cells  in 
contention  of  hiding  the  target  are  examined  and  ranked.  If  any  area-cell,  when 
compared  to  the  area-cell  of  maximum  likelihood,  fails  to  meet  a  certain  threshold,  that 
area-cell  is  eliminated  pennanently  from  contention.  The  process  continues  until  only 
one  area-cell  remains  in  the  pool  of  candidates,  and  that  cell  is  declared  to  contain  the 
target.  We  show  that  our  sequential  models  provide  efficient  solutions  to  this  class  of 
problem,  while  guaranteeing  to  meet  the  user-prescribed  error  tolerances.  In  particular, 
we  show  our  procedure  not  only  outperfonns  a  typical  sensible  approach  that  uses  the 
same  expected  number  of  investigations;  indeed  the  sensible  approach  fails  to  meet  the 
error  tolerance.  As  with  our  adaptive  model  for  the  first  situation,  the  cost-savings  to  the 
sensor  operator  and  operational  planner  when  implementing  this  sequential  eliminating 
procedure  is  evident;  scarce  resources  may  be  more  readily  available  to  perform  other 
critical  sensing  tasks. 
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I.  INTRODUCTION 


Today’s  operational  planners  and  sensor  operators  are  challenged  with  scarce 
resources  in  terms  of  both  time  and  sensor  assets.  One  may  argue  that  recent 
advancements  in  technology  have  slowed  the  growth  of  innovative  employment  tactics, 
techniques,  and  procedures  (TTPs),  making  sensor  operators  increasingly  more 
dependent  upon  that  technology.  Defense  planners  must  strive  to  develop  and 
incorporate  new,  efficient  procedures  to  allocate  scarce  resources  in  many  different 
complex  environments.  Any  efficiency  that  can  be  gained,  however  small,  may  have  a 
compound  effect  over  time  on  overall  combat  readiness,  by  freeing  up  precious  assets  to 
perform  other  time-sensitive,  critical  sensing  actions. 

In  this  thesis,  we  consider  two  particular  search  effort  allocation  situations  of 
operational  interest.  We  refer  to  these  as  the  Single  Markov  Target  ( SMT)  model,  and  the 
Single  Static  Target  (SST)  model.  We  begin  by  describing  some  characteristics  common 
to  both  models,  then  follow  with  discussions  of  the  background  and  problem  statements, 
objectives,  and  scope  and  limitations  for  the  SMT  and  SST  models,  treated  in  Section  B 
and  Section  C,  respectively. 

A.  MODEL  COMMONALITIES 

Both  the  SMT  and  SST  models  allow  us  to  contend  with  the  fact  that  search 
sensors  are  imperfect;  i.e.,  they  are  subject  to  declaring  a  target  present  when  it  is  in  fact 
absent  (false-positive  error),  as  well  as  to  declaring  a  target  absent  when  it  is  in  fact 
present  (false-negative  error).  The  Area  of  Operational  Interest  (AOI)  for  both  models  is 
comprised  of  a  grid  of  discrete,  non-overlapping  area-cells,  each  cell  having  its  own 
associated  values  for  sensor  error  rates.  These  area-cells  might  be  defined  by  geo¬ 
political  borders,  terrain  features,  or  some  arbitrary  grid  system  of  tactical  significance  to 
the  operator,  and  need  not  be  uniform  in  size  nor  shape.  Both  models  involve  a  single 
target  of  interest  (TOI),  located  somewhere  within  the  AOI.  The  TOI  is  non-reactive; 
i.e.,  it  does  change  its  pattern  of  behavior  in  response  to  any  sensing  action.  We  treat 

time  in  both  of  these  models  in  tenns  of  discrete  time-steps.  The  operator  makes  one 
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investigation  into  one  area-cell  per  time-step.  Both  models  deal  with  the  allocation  of 
search  sensors,  which  implies  that  we  are  concerned  with  the  placement  of  sensors,  and 
that  those  sensors  are  not  restricted  to  follow  any  particular  path.  In  contrast,  a  search 
path  problem  might  impose  such  a  restriction,  say,  e.g.,  only  immediately  adjacent  area- 
cells  may  be  investigated  on  subsequent  time-steps.  It  may  be  assumed  either  that 
sufficient  time  exists  between  investigations  so  that  a  single  sensor  may  be  re -positioned 
to  any  other  area-cell,  or  that  each  area-cell  contains  one  pre-positioned  sensor,  but  the 
operator  may  process  only  one  of  those  sensors  in  any  time-step. 

In  summary,  both  SMT  and  SST  are  discrete-time,  discrete  space,  single  non¬ 
reactive  target,  single  searcher,  path  unconstrained,  imperfect  search  sensor  allocation 
models. 

B.  SINGLE  MARKOV  TARGET  (SMT)  MODEL 

1.  Problem  Statement 

Consider  an  AOI,  in  which  a  TOI  is  known  to  be  operating.  The  TOI  could  be  a 
convoy,  a  vehicle,  or  an  individual  insurgent.  Assume  that,  based  on  intelligence  data 
and  social  theory,  this  particular  TOI  is  subject  to  movement  in  a  Markovian  fashion. 
That  is  to  say,  at  each  time  step,  the  TOI  moves  randomly  according  to  some  probability 
mass  function  (pmf),  which  may  depend  on  a  finite  number  of  current  and  past  locations. 
An  area-cell  is  considered  to  be  determined  when  enough  evidence  exists  for  the  sensor 
operator  to  declare  that  a  target  is  either  present  or  absent  in  that  cell.  The  resultant 
operational  problem  is  summarized:  how  to  estimate  the  steady-state  distribution  of  the 
TOI  (whose  transition  matrix  and  resultant  steady-state  distribution  clearly  are  unknown 
to  the  searcher)  adequately,  based  solely  on  noisy  observations  from  imperfect  sensors. 

2.  Objective 

The  Strong  Law  of  Large  Numbers  suggests  that  any  search  effort  scheme  that 
allocates  a  positive  allocation  of  effort  to  every  possible  area-cell  leads  to  estimates  that 
converge  upon  the  true  steady  state  distribution  of  the  target  (see  Chapter  III  for  a  more 
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thorough  discussion  of  this).  Here,  the  objective  is  to  detennine  an  allocation  scheme 
that  converges  quickly,  while  displaying  an  improved  error  decay  rate  (where  the  error  is 
an  incorrectly  determined  area-cell)  when  compared  to  other  schemes  for  cases  in  which 
the  available  number  of  search  opportunities — the  search  budget — is  large. 

3.  Scope  and  Limitations 

For  the  scenario  used  in  this  portion  of  the  thesis,  we  consider  only  one  single 
target,  whose  movement  is  characterized  as  Markovian.  A  target  that  moves  according  to 
some  other  scheme  would  not  be  appropriate  for  this  model.  We  do  not  consider  an 
intelligent  or  reactive  target.  Additionally,  we  consider  only  a  single  sensor  (or  multiple 
sensors  subject  to  the  constraint  that  only  one  sensor  may  be  used  at  a  particular  time- 
step);  therefore,  we  do  not  consider  cooperation  among  sensors.  It  is  assumed  that  sensor 
sensitivity  and  specificity  (see  Chapter  II  for  the  associated  definitions)  for  each  area  are 
known  values.  In  reality,  it  is  likely  that  these  values  would  be  noisy;  the  manufacturer 
might  provide  to  the  operator  their  expected  values  as  published  specifications,  or 
perhaps  the  operator  might  derive  them  using  some  form  of  tactical  decision  aid.  Our 
model  does  not  take  into  account  that  the  sensitivity  and  specificity  values  might  be 
correlated  with  the  number  of  looks;  e.g.,  sensors  with  recognition  algorithms  are  likely 
to  exhibit  some  form  of  learning  behavior,  with  error  rates  decreasing  with  the  number  of 
observations.  Further,  we  assume  that  the  operator’s  search  budget  is  large;  otherwise,  a 
dynamic  programming  approach  might  be  suitable  to  this  particular  problem.  It  will  be 
shown  that  for  relatively  small  search  budgets,  the  adaptive  algorithm  we  propose  is  not 
the  best  choice. 

C.  SINGLE  STATIC  TARGET  (SST)  MODEL 

1.  Problem  Statement 

Again,  consider  an  AOI  and  a  particular  TOI  for  which  the  operator  is  searching. 
This  time,  however,  we  are  concerned  with  a  TOI  that  is  stationary,  or  static,  somewhere 
within  the  AOI.  Operational  TOIs  for  this  scenario  might  include,  for  example,  enemy 
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insurgents  in  hiding  or  a  downed  friendly  aircraft.  A  sequential  eliminating  procedure  is 
well  suited  for  this  particular  application.  In  practice,  any  situation  where  the  TOI 
remains  static  in  a  timescale  that  is  larger  than  that  of  the  search  process  fits  the 
framework  of  this  model.  A  sequential  eliminating  procedure  attempts  to  isolate,  from 
among  several  candidate  systems,  one  particular  desired  system — the  “objective.”  During 
a  particular  stage  of  a  sequential  eliminating  procedure,  all  candidate  systems  are 
examined  and  ranked  in  order  of  their  likelihood  of  being  the  objective.  Each  system  is 
then  compared  to  the  most  likely  objective — the  system  ranked  highest  for  that  stage — by 
means  of  the  ratio  of  their  likelihood  ratios  (i.e.,  their  odds  ratio).  Any  system  whose 
odds  ratio  fails  to  meet  a  certain  threshold  (which  we  define  in  Chapter  III)  is 
permanently  removed  ( eliminated)  from  the  set  of  candidates.  If  all  systems  meet  the 
threshold  during  a  particular  stage,  then  all  those  systems  remain  in  the  set  of  candidates 
(this  is  referred  to  as  the  continuation  region).  The  procedure  advances  to  the  next  stage, 
using  the  updated  candidate  set.  The  process  continues  until  only  one  system  remains  in 
the  set  of  candidates,  and  that  system  is  declared  the  winner  (in  our  case,  the  systems  are 
the  area-cells,  and  the  winner  is  the  area-cell  containing  the  TOI).  The  operational 
dilemma  for  this  scenario  is  to  make,  as  quickly  as  possible,  a  proper  determination  of 
TOI  location,  again  based  solely  on  the  noisy  sensor  observations. 

2.  Objective 

For  the  SST  model,  we  set  forth  to  develop  efficient  criteria  for  the  sequential 
eliminating  procedure,  which,  when  followed,  result  in  determination  of  TOI  location 
meeting  certain  operator-defined  error  tolerances. 

3.  Scope  and  Limitations 

We  once  again  restrict  our  study  to  the  case  of  a  single  target  and  a  single  sensor.1 
For  the  SST  model,  however,  the  TOI  is  assumed  to  be  stationary,  at  least  for  the  duration 
of  the  search  period.  The  same  assumptions  made  in  the  SMT  model  regarding  sensor 

1  As  an  exception,  we  treat  a  brief  introductory  case  (case  1)  in  which  each  area-cell  either  contains  or 
does  not  contain  a  TOI,  independent  of  all  other  area-cells. 
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sensitivity  and  specificity  are  relevant  to  the  SST  model,  namely  that  the  values  are 
treated  as  fixed  for  each  area,  and  that  there  is  no  correlation  among  sensor  observations. 
Other  assumptions  made  without  loss  of  generality  will  be  noted  in  Chapter  III. 
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II.  LITERATURE  REVIEW 


This  chapter  discusses  previous  literature  and  research  relevant  to  this  thesis,  and 
consists  of  four  sections.  We  begin  with  an  overview  of  some  types  of  sensors  currently 
in  use  or  in  development  for  unmanned  aerial  systems  (UAS),  as  well  as  a  brief  survey  of 
some  research  and  literature  related  to  search  theory.  An  introduction  to  some  ideas  in 
the  field  of  stochastic  search  and  optimization  follows,  with  special  attention  paid  to  the 
concept  of  stochastic  approximation.  We  continue  with  a  discussion  on  sequential 
analysis  as  a  primer  for  the  second  model  of  this  thesis.  Finally,  we  present  some 
conclusions  and  a  justification  for  the  research  of  this  thesis. 

A.  BACKGROUND 

1.  Sensors  and  Complex  Environments 

The  nature  of  recent  conflicts  has  imparted  a  two-fold  effect  on  the  employment 
of  UASs.  First,  it  has  caused  the  primary  areas  of  operation  to  migrate  into  areas  in 
which  it  is  difficult  to  operate.  Second,  it  has  placed  increased  importance  on  the 
technological  development  of  airborne  Intelligence,  Surveillance,  and  Reconnaissance 
(ISR)  sensors  to  counter  both  asymmetric  and  conventional  threats.  Primary  missions 
areas  for  tactical  UASs  in  Iraq  and  Afghanistan  include  point  surveillance,  target 
following,  area  search,  route  reconnaissance,  and  Improvised  Explosive  Device  (IED) 
detection  (Owen,  Martin,  &  Carriger,  2005).  Missions  flown  by  Pioneer,  Scan  Eagle,  and 
Shadow  UASs  normally  service  a  list  of  targets  provided  by  intelligence  units.  These 
target  lists  are  typically  comprised  of,  for  example,  suspected  insurgent  safe  houses, 
suspected  weapons  caches  and  mortar  points  of  origin  as  well  as  direct  support  for  raids, 
patrols,  convoys  and  other  operations  (Reber,  2007).  Additionally,  recent  research  efforts 
have  explored  the  use  of  UASs  in  the  detection  of  possible  chemical  or  biological  plumes 
(Scheldt,  2008).  Modern  conventional  and  emergent  asymmetric  threats  have  indeed 
shaped  a  challenging  battle-space  to  frame  these  missions. 
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Geographic  areas  that  operators  consider  inhospitable  or  undesirable  for  the 
employment  of  UASs  include  terrain  prevalent  in  current  regions  of  major  conflict.  A 
geometrically  diverse  urban  canopy  and  a  cluttered,  mountainous  border  crossing  are  two 
examples  of  areas  that  many  consider  exceptionally  challenging  for  the  operation  of 
UASs.  We  refer  to  these  locations  collectively  as  complex  environments.  The  challenges 
faced  by  operators  and  customers  of  UASs  associated  with  these  complex  environments 
include,  for  example,  variable  levels  of  autonomy,  collision  avoidance,  wind  gusts  and 
turbulence,  unreliable  wireless  communications,  stealth,  power  and  energy  management, 
and  portability  (Dodd  &  Apopei,  2007). 

An  elementary  characterization  of  UAS-bome  sensors  in  operation  and  under 
development  today  is  via  their  sensitivity  and  specificity,  two  terms  adopted  from  the 
binary  classification  test  as  measures  of  performance  for  discriminatory  sensors  (Kress, 
Szechtman,  and  Jones,  2008).  The  sensitivity  of  a  sensor  is  a  measure  of  its  ability  to 
correctly  detect  a  real  target,  whereas  a  sensor’s  specificity  is  its  ability  to  correctly  reject 
(i.e.,  not  detect)  everything  that  is  not  a  target  of  interest.  Both  of  these  characteristics 
are  measured  as  probabilities,  and  lead  to  complementary  sensor  error  rates.  False 
negative  (miss)  rates  are  expressed  algebraically  as  1 -sensitivity,  and  false  positive  (false 
alarm)  rates  are  expressed  as  1 -specificity.  Examples  of  sensors  that  may  be 
characterized  by  both  sensitivity  and  specificity  include  chemical  and  biological  plume 
detectors,  Inverse  Synthetic  Aperture  Radar  (ISAR),  Synthetic  Aperture  Radar  (SAR), 
multispectral  and  hyperspectral  imaging  sensors,  and  other  Coherent  Change  Detection 
(CCD)  and  recognition-based  sensors  (Suter,  2005). 

Determining  a  sensor’s  sensitivity  and  specificity  for  any  region  is  not  a  trivial 
matter.  Environmental  factors  such  as  wind,  temperature,  humidity,  ambient  light,  and 
atmospherics,  as  well  as  physical  makeup  of  the  target  compared  to  its  surroundings, 
target  and  searcher  motion,  and  line-of-sight  considerations  are  all  capable  of  affecting 
sensor  performance  (Calhoun,  et  ah,  2007).  For  certain  Human-in-the-Loop  (HITL) 
systems,  the  added  complicating  factor  of  operator  recognition  differential  (RD)  is  often 
subjective  and  very  difficult  to  quantify.  Additionally,  certain  sensors  are  susceptible  to 
performance  degradation  over  time,  possibly  decreasing  both  sensitivity  and  specificity. 
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2. 


Search  Theory  and  Mission  Planning 


Since  operators  can  afford  neither  infinite  dwell  time  nor  infinite  sensors,  a  model 
must  be  developed  that  allocates  assets,  constrained  by  a  particular  search  budget,  in  a 
manner  that  optimizes  certain  measures  of  performance  and  measures  of  effectiveness 
(MOP/MOE).  Much  of  the  research  done  to  date  on  sensor  allocation  has  its  roots  in  the 
ideas  put  forth  by  Koopman  in  his  pioneering  report,  Search  and  Screening ,  penned  in 
1946  and  declassified  in  1958. 

Benkoski,  Monticino,  and  Weisinger  (1991)  give  a  survey  of  literature  published 
on  search  theory  up  until  1991.  Their  discussion  covers  problems  with  non-cooperative 
targets,  as  opposed  to  cooperative  or  rendezvous  problems.  The  class  of  non-cooperative 
target  problems  includes  both  those  having  to  do  with  passive  targets  (one-sided  search) 
and  those  concerned  with  evasive  targets  (search  games).  They  break  down  search 
problems  by  time  and  space  (discrete  versus  continuous),  target  motion  (stationary  versus 
moving),  and  constraints  on  searcher  motion  (paths  versus  search  effort  allocation).  The 
authors  also  discuss  additional  extensions,  including  multiple  searchers  and  targets, 
uncertain  detection  probabilities,  and  varying  objective  functions. 

One  of  the  most  widely  cited  treatments,  and  one  whose  motivation  relates  most 
closely  to  the  focus  of  this  thesis,  is  that  proposed  by  Washburn  (1983).  His  study 
concerned  the  application  of  an  iterative  Forward  and  Backward  (FAB)  algorithm, 
originally  put  forth  by  Brown  (1977),  to  compute  optimal  (in  the  sense  of  maximizing 
probability  of  target  detection)  search  plans  when  the  motion  of  the  target  is  modeled  by 
a  discrete  space  and  time  Markov  chain  with  known  transition  matrix.  The  FAB 
algorithm  is  also  the  tool  of  choice  for  Dambreville  and  Le  Cadre  (1999)  to  allocate 
search  effort  in  the  case  where  search  assets  renew  with  generalized  linear  constraints. 
Oshumi  (1991)  tackles  a  similar  problem  to  that  of  Washburn,  but  in  continuous  time 
where  target  motion  is  described  by  stochastic  differential  equations,  rather  than  by  a 
Markov  process. 

Prior  to  Washburn’s  work,  moving  target  problems  could  only  be  solved  in 
certain  cases.  One  particular  case  is  that  in  which  target  motion  is  conditionally 
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deterministic  with  a  factorable  Jacobian.  In  this  case,  one  can  reduce  the  to  a  stationary- 
target  problem  and  solve  it  via  stationary-target  techniques  (Stone,  1977;  Pursiheimo, 
1976;  and  Iida,  1972).  The  other  case  is  one  in  which  the  number  of  area-cells  is  small, 
despite  a  priori  knowledge  of  the  associated  optimality  conditions  (Lions,  1971; 
Heilman,  1972;  Saretsalo,  1973;  Pollock,  1970;  &  Dobbie,  1974). 

Taking  the  idea  of  a  non-cooperative  target  a  step  further,  some  authors  have 
considered  targets  that  take  evasive  action.  Such  problems  rarely  lend  themselves  to 
analytic  solutions;  in  such  cases,  simulation  may  provide  a  suitable  alternative 
(Washburn,  1989).  Another  approach  utilizes  the  minimax  strategy  of  game  theory  to 
maximize  the  searcher’s  probability  of  detecting  an  evasive  target  (Dambreville  and  Le 
Cadre,  2001).  One  possible  drawback  to  the  minimax  strategy  is  the  potentially 
prohibitive  computational  cost  involved  upon  introduction  of  varied  strategies  for 
multiple  searchers  or  targets.  Carl  (2003)  chose  the  former  approach  in  his  thesis 
studying  the  search  for  German  U-Boats  in  the  Bay  of  Biscay  during  World  War  II,  using 
agent-based  simulation  to  evaluate  Allied  search  plans. 

DelBalzo  and  Hemsteter  (2002)  present  a  genetic  algorithm  approach  to  the 
evasive  target  problem.  They  show  that,  in  general,  an  evasively  maneuvering  target  as 
compared  to  a  randomly  patrolling  target  reduces  the  cumulative  detection  probability 
(CDP)  in  sonar  search  dramatically,  since  counter-detection  ranges  are  typically  greater 
than  detection  ranges.  Their  analysis  covers  several  combinations  of  platfonns  and 
sensors  in  a  simulated  environment.  Their  Genetic  Range-dependent  Algorithm  for 
Search  Planning  (GRASP)  and  associated  joint  tactics  exploit  evasive  target  maneuvers 
and  provide  increased  CDP  over  non-joint  tactics. 

Whereas  the  typical  objective  of  a  detection  search  is  to  maximize  probability  of 
detection,  the  objective  of  a  surveillance  search  is  to  maximize  the  probability  of 
detecting  the  target  at  a  specific  time  or  in  a  specific  region.  Similarly,  the  object  in 
whereabouts  search  is  to  localize  a  target  to  within  one  of  a  finite  number  of  cells 
(Benkoski,  Monticino,  &  Weisinger,  1991).  In  this  case,  searcher  success  is  achieved  by 
either  detecting  the  target,  or,  if  the  target  is  not  detected,  by  correctly  guessing  the  cell 

containing  the  target.  Tognetti  (1968)  and  Kadane  (1971)  treat  the  scenario  of 
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whereabouts  search  against  a  stationary  target.  By  showing  that  solving  a  whereabouts 
search  is  equivalent  to  solving  a  finite  number  of  optimal  detection  search  problems, 
Stone  and  Kadane  (1981)  make  general  the  earlier  results  to  encompass  the  moving  target 
problem.  Finally,  Tierney  and  Kadane  (1983)  provide  necessary  optimality  conditions 
and  an  algorithm  that  constructs  search  plans  for  the  surveillance  search  problem  against 
a  Markovian  target  with  known  transition  matrix. 

Dell,  Eagle,  Santos,  and  Martins  (1996)  formulate  a  discrete  time,  path 
optimization  problem  for  multiple  searchers.  They  utilize  a  branch- and-bound  algorithm 
and  six  heuristics  for  solving  such  problems.  In  a  related  approach,  Sato  and  Royset 
(2008)  develop  a  specialized  branch-and-bound  algorithm  and  a  Lagrangian  relaxation- 
based  bounding  technique  to  solve  problems  where  the  searcher  is  constrained  by 
consumption  limits  of  several  resources.  In  their  problem,  the  searcher  knows  both  the 
initial  target  distribution  and  its  Markovian  transition  matrix. 

The  problem  addressed  by  Zhang  and  Chen  (2006)  deals  with  multiple  imperfect 
sensor  allocation  against  multiple  targets  in  discrete  time,  where  the  overall  goal  is  to 
minimize  target  location  error.  In  their  model,  estimated  target  position  is  represented 
through  a  probability  grid  updated  dynamically  by  belief  states  based  on  sensor  input. 
Sensor  errors  are  inherent  in  the  Gaussian  signal  strength  inputs  of  the  sensors;  however, 
they  do  not  deal  directly  with  sensitivity  and  specificity  in  calculating  allocations  that  are 
optimal  in  the  sense  of  minimizing  incorrect  determinations. 

In  his  thesis,  Lohr  (1992)  discusses  Area  Motion  Search,  a  hybrid  of  classical 
search  and  detection  theory  models  of  Exhaustive  Search  and  Random  Search.  In  this 
model,  target  motion  is  random  in  continuous  time,  searcher  motion  is  systematic  in 
continuous  time,  and  detection  opportunities  in  non-overlapping  time  periods  are 
probabilistically  independent.  Again,  sensor  error  rates  are  not  considered. 

Peot  et  al.  (2005)  suggest  a  probabilistic  roadmap  approach  to  the  urban  UAS 
routing  problem  which  maximizes  the  utility  of  sensing  actions  for  a  given  collection 
strategy,  where  the  benefit  is  modeled  as  the  cumulative  probability  of  detection, 
recognition,  or  identification  during  the  period  that  the  target  is  observed.  A  penalty  cost 
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due  to  exposure  to  threats  or  navigational  hazards  in  the  environment  offsets  the  benefit. 
While  the  robust  tool  is  one  that  maximizes  sensing  actions  with  respect  to  sensor  dwell, 
communications,  Line  of  Sight  (LOS),  and  UAS  flight  kinematic  constraints,  the  authors 
do  not  address  sensitivity  and  specificity  of  the  sensors. 

Yan  and  Blankenship  (1987)  propose  a  list  of  tasks  for  a  detection  search,  and 
emphasize  the  non- trivial  nature  of  each  step.  The  steps  they  outline  are  highly 
applicable  to  our  study: 

1 .  Compute  a  prior  distribution  of  target  location. 

2.  Obtain  a  good  estimate  of  sensor  capabilities. 

3 .  Determine  a  detection  (misdetection)  function. 

4.  Develop  a  search  plan  and  estimate  its  success  probability. 

5 .  Update  the  posterior  target  distribution  from  search  feedback. 

6.  Evaluate  search  effectiveness  (Yan  and  Blankenship,  1987). 

Step  5  is  often  referred  to  as  the  Search  Control  Problem,  and,  for  Yan  and 
Blankenship,  involves  determining  a  search  path  that  minimizes  the  target  survivability 
up  to  a  certain  time.  They  solve  the  problem  on  a  simplified  search  model,  in  discrete 
time  and  space,  by  embedding  the  Dual  Estimation  Problem  (Yan  &  Blankenship,  June 
1987)  into  their  Ordered  Search  Algorithm,  a  best- first  search  algorithm.  The  resultant 
Optimal  Detection  Search  Algorithm  (ODSA)  describes  the  real  model  more  precisely. 
ODSA  not  only  updates  target  distribution  at  the  beginning  of  each  time  step  to  hone  the 
accuracy  of  the  Search  Estimation  Problem  (see  Step  4  above);  it  also  finds  an  optimal 
path  of  the  Search  Control  Problem.  By  applying  an  efficient  heuristic  to  the  Ordered 
Search  Algorithm,  Yan  and  Blankenship  show  convergence  to  optimal  paths,  while 
expanding  only  about  one  ninth  of  the  nodes  expanded  by  an  exhaustive  search,  thus 
fulfilling  the  evaluation  of  search  effectiveness  (see  Step  6  above). 

As  noted  in  Step  1 ,  the  ability  to  construct  a  prior  distribution  of  target  position  is 
of  interest.  One  possible  source  for  determining  the  initial  target  probability  mass 
function  is  a  tool  known  as  Threat  Mapper.  Riese  (2006)  developed  the  software  tool, 
which  leverages  the  robustness  and  availability  of  geospatial  infonnation  systems  (GIS) 

and  fuses  historical  data  to  aid  analysts  and  forces  in  making  spatial  forecasts  to  support 
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intelligence  operations.  The  planner  enters  locations  of  past  events  of  interest  (see  Figure 
1,  wherein  historical  events  are  represented  by  red  triangles),  and  areas  of  spatial 
similarity  are  determined  and  used  as  a  forecast  for  future  events.  The  output  is  a  color- 
coded  map  of  absolute  spatial  similarity,  as  determined  by  user  input  characteristics  (see 
Figure  2,  where  red  areas  are  of  high  likelihood,  and  blue  areas  are  of  low  likelihood), 
and  can  be  normalized  to  create  what  can  be  used  as  a  probability  map. 


Figure  1 .  Example  Threat  Mapper  input  (From  Riese,  2008). 
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Figure  2.  Example  Threat  Mapper  output  (From  Riese,  2008). 


B.  STOCHASTIC  SEARCH  AND  OPTIMIZATION 

Whereas  much  of  the  previous  research  associated  with  the  UAS  routing  problem 
has  focused  on  classical  search  theory  as  explained  in  Section  C  of  this  chapter,  the  noise 
and  uncertainty  associated  with  the  complex  environments  described  above  necessitate  a 
robust  tool.  Given  the  complexity  of  many  real-world  problems  faced  by  industry  and 
government  today,  along  with  the  inherent  uncertainty  in  the  information  that  might  be 
available  to  the  problem  solver,  stochastic  search  and  optimization  models  have  been 


14 


playing  a  growing  role  (Spall,  2003). 2  In  stochastic  optimization,  there  are  generally  two 
possible  sources  of  uncertainty.  In  the  first  case,  random  noise  is  present  in  the 
measurement  of  either  a  loss  function,  which  is  a  scalar  measurement  summarizing  the 
performance  of  the  system  for  a  given  value  of  the  vector  of  the  “adjustables,”  or  its 
gradient  function.  In  the  second  case,  there  is  a  random  choice  made  in  the  search 
direction  as  the  algorithm  iterates  toward  a  solution.  For  a  given  problem,  there  is  also 
the  possibility  that  noise  is  present  due  to  both  of  the  aforementioned  cases.  In  the 
problem  presented  in  this  thesis,  the  sources  of  noise  are  the  misclassification  errors 
inherent  to  the  sensors,  and  the  uncertainty  associated  with  the  movement  of  a  target. 

One  of  the  cornerstones  of  stochastic  search  and  optimization  is  the  idea  of 
stochastic  approximation  (SA)  (see  Kushner  &  Yin,  2003).  SA  algorithms  are  iterative 
methods  of  finding  extremes  or  roots  of  functions  whose  values  cannot  be  calculated 
directly,  but  instead  must  be  approximated  based  on  noisy  observed  values.  For  example, 
let  us  start  by  considering  a  real  function  g,  and  suppose  the  goal  is  to  find  the  value  0 
such  that  g[6* )  =  0  .  Assume,  for  simplicity,  that  g(0)  >  0  for  6  <  6  and  that  g{0)  <  0 

for  G>6* .  The  recursive  procedure  is 

0n+l=0n+eg(e«) 

for  e  >  0  .  If  0n  <  6  then  g(0„)  >  0,  meaning  that  0n+l  >  0n ,  and  hence  0n+l  moves  in 
towards  0  .  If  0n  >  0  then  g(0H )  <  0 ,  so  that  0n+l  <  0„ ,  and  0n+1  moves  to  the  left, 
approaching  0 * . 

Robbins  and  Monro  (1951)  extended  the  above  procedure  to  the  case  where  the 
function  g  is  unknown,  but  can  be  estimated  via  noisy  observations.  The  recursion  is 

&n+:  1  =  +£,J„ 

where  £„>0,  XL^»=GO’  anc*  are  n°isy  observations  of  g(0„)  .  More 

precisely,  Y=g(0n)  +  8,  with  E[Yn\Yj,0i  ,  i  <  n,  0n  ]  =  g  {0n ) ,  and  where  the  error  term  8 


-  Here,  the  term  “search”  is  referring  to  the  algorithmic  approach  to  finding  an  optimal  solution,  as 
opposed  to  attempting  to  locate  a  target,  which  has  been  the  definition  referred  to  thus  far. 
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has  finite  variance.  (The  conditioning  elements,  Yj,6j  ,i  <  n,  6n ,  comprise  the  history  up  to 
stage  n .)  In  this  case,  the  recursion  can  be  written  as 

^=en+eng(On)  +  sn-S, 

suggesting  that  the  effect  of  the  error  term  8  vanishes  as  n  — >  oo  due  to  the  finite  error 
variance  and  the  conditions  imposed  on  sn . 

Soon  after  Robbins  and  Monro  (1951)  introduced  their  algorithm,  Kiefer  and 
Wolfowitz  (1952)  built  upon  it  by  injecting  a  second  sequence  of  positive  step  sizes, 
which  are  used  to  estimate  the  derivative  of  the  function  of  true  values  via  the  difference 
between  the  observed  values  and  the  new  step  sizes.  They  showed  that  if  both  sequences 
of  step  sizes  fulfill  certain  bounds,  and  the  functions  of  the  noisy  and  true  values  satisfy 
certain  conditions,  then  the  observed  values  converge  in  probability  to  the  true  value.  It 
is  a  concept  similar  to  that  of  the  Robbins-Monro  and  Kiefer-Wolfowitz  algorithms  that 
is  at  the  heart  of  the  stochastic  approximation  algorithm  used  in  the  SMT  model  of  this 
thesis. 


C.  SEQUENTIAL  ANALYSIS 

In  the  second  situation  of  this  thesis,  the  operator  is  concerned  with  quickly 
determining  the  presence  or  absence  of  a  target  subject  to  certain  type-I  and  type-II  error 
tolerances,  which  are  specified  by  the  operator.  The  goal  is  to  stop  the  search  in  the  least 
expected  amount  of  time,  subject  to  the  error  bounds.  This  problem  is  intrinsically 
sequential,  as  it  deals  with  the  fixed  precision  estimation  of  a  parameter  in  the  presence  of 
an  unknown  nuisance  parameter.  The  theory  behind  sequential  analysis  is  therefore  a 
well-suited  solution  approach  in  this  case. 

Siegmund  (1985)  is  the  classic  reference  in  this  field,  and  deals  primarily  with 
sequential  hypothesis  testing  and  related  problems  of  estimation.  In  many  of  these  cases, 
a  fixed  sample  solution  exists  and  one  employs  sequential  methods  in  order  to  achieve 
some  greater  efficiency  in  the  solution.  For  example,  consider  the  case  where  one  wishes 
to  infer,  on  the  basis  of  a  random  sample,  whether  the  proportion  of  defective  items  in  a 
large  batch  exceeds  some  value  p0 .  Assume  that  the  inference  will  be  based  on  the 
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number  Sm  of  defectives  in  a  random  sample  of  size  m.  If  m  is  a  small  proportion  of  the 
batch  size,  then  Sm  has  (approximately)  a  binomial  distribution  with  mean  mp,  where  p  is 
the  true  proportion  of  defectives  in  the  batch,  and  a  reasonable  rule  to  test  the  hypothesis 

7i[)\p<p0  against  7 ik:p>  p0 


is  to 


reject  7^  if  Sm  >  r 

for  some  constant  r.  If  the  sample  is  drawn  sequentially,  and  for  some  value  k  less  than 
m,  the  value  of  Sk  already  equals  r,  one  could  stop  sampling  immediately  and  reject  'HM . 


More  formally,  let  T  denote  the  smallest  value  of  k  for  which  Sk  =  r  and  put 

T'  =  min  ( T jn) .  Consider  the  procedure  that  stops  sampling  at  the  random  time  T  and 

decides  that  p  >  p0  if  and  only  if  T  <  m.  If  one  considers  these  procedures  as  tests  of 

against  77,,  their  rejection  regions,  namely  j  T  <  /?/  j  and  {  Sm  >  r  j,  are  the  same 

events,  and  hence  the  two  tests  have  the  same  power  function.  Since  the  test  which  stops 
at  random  time  T'  never  takes  more  observations  and  may  take  fewer  observations  than 
the  fixed  sample  test,  it  has  a  reasonable  claim  to  be  regarded  as  more  efficient 
(Siegmund,  1985,  p.  2).  Siegmund  acknowledges  additionally  that  sequential  methods 
are  a  natural  choice  for  parameter  estimation  problems,  such  as  the  SST  model  of  this 
thesis. 


Malone  (2004)  treats  ranking  and  selection  procedures  for  both  Bernoulli  and 
multinomial  systems.  These  Bernoulli  ranking  and  selection  procedures  are  related  to  our 
problem,  since  the  sensors  sample  from  a  Bernoulli  distribution  for  each  system  (area¬ 
cell),  with  parameter  that  depends  on  whether  the  target  is  present  or  absent  in  that  area¬ 
cell.  In  her  thesis,  however,  each  system  has  unknown  Bernoulli  parameter  ,  and 

the  goal  is  to  select  the  system  with  the  largest  a .  She  applies  fully  sequential 
procedures  to  Bernoulli  data  for  tenninating  solutions,  and  significant  savings  in  total 
observations  are  realized  for  two  to  five  systems,  when  one  desires  to  detect  small 
differences  between  competing  systems. 
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Wieland  and  Nelson  (2004)  present  a  sequential,  eliminating  procedure  for 
selecting  the  best  system  in  a  single-factor  Bernoulli-response  experiment  with  an  odds- 
ratio  indifference  zone.  Similar  to  Malone,  in  their  case,  “best”  refers  to  the  system  with 
largest  probability  of  success  on  a  given  trial.  Recall,  in  subtle  contrast,  the  problem 
considered  in  this  thesis.  Consider  an  AOI  comprised  of  m  area-cells, 
labeled ^Cj,...,  ACm.  Assume,  without  loss  of  generality,  that  a  target  is  present  within 

A C, .  Let  ai  be  the  sensor’s  sensitivity  for  cell  i,  and  let  1  -bt  be  the  specificity  of  the 
sensor  for  cell  i  (see  Chapter  III  for  definitions  of  these  terms).  In  A  C, ,  we  sample  from 
a  Bernoulli  distribution  with  parameter  av  In  A  Cj ,  i  =  2,...,m,  we  sample  from  a 
Bernoulli  distribution  with  parameter  h  .  Given  that  ai  and  bt  are  known  for  i  =  1 

the  problem  is  that  of  determining  the  area-cell  from  which  samples  are  drawn  from  a 
Bernoulli  distribution  with  parameter  a  . 

D.  LITERATURE  REVIEW  CONCLUSIONS 

Upon  review,  the  nature  of  recent  sensor  employment  in  complex  environments, 
with  consideration  to  the  characteristics  of  emergent  sensor  technology,  necessitates 
models  that  provide  efficient  allocation  of  scarce  sensors  in  order  to  provide  sensor 
operators  and  operational  planners  with  the  availability  and  flexibility  required  on  today’s 
battlefields.  We  acknowledge  that  significant  research  has  been  conducted  on  search 
effort  allocation  against  a  moving  target.  Nevertheless,  we  hope  to  offer  genuinely  new 
insight  by  framing  the  problem  in  this  operational  context,  in  discrete  time  and  space, 
while  considering  both  sensor  sensitivity  and  specificity,  and  through  use  of  an  adaptive 
algorithm  based  upon  stochastic  approximation  to  determine  steady-state  location 
distribution  of  a  Markovian  target  with  unknown  transition  matrix.  Additionally,  the 
research  to  date  in  the  field  of  sequential  analysis  has  only  considered  eliminating 
procedures  that  attempt  to  find  the  “best”  Bernoulli  system — typically  the  one  with 
maximum  (unknown)  parameter  value.  In  contrast,  we  wish  to  find  the  system  whose 
parameter  is  most  likely  to  be  equal  to  a  known  value,  which  is  different  for  each  system 
and  may  not  be  the  maximum.  This  is  an  operationally  relevant  problem  for  static 
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targets,  which  we  set  forth  to  solve  efficiently  with  a  sequential  eliminating  procedure 
that  is  guaranteed  to  meet  type-I  and  type-II  error  thresholds. 
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III.  MODEL  DEVELOPMENT 


In  this  chapter,  we  discuss  the  development  and  fonnulation  of  the  models  used  in 
this  thesis.  We  start  by  introducing  some  theory  and  details  of  the  SMT  model,  then 
proceed  to  outline  the  stochastic  approximation  algorithm  used  to  provide  solutions.  We 
then  follow  a  similar  pattern  for  the  SST  model  and  its  sequential  eliminating  procedure. 

A.  SMT  MODEL 

1.  Basic  Framework 

Suppose  that  the  area  of  operational  interest  (AOI)  is  partitioned  into  m  area-cells. 
Let  n  =  (nl be  the  steady-state  distribution  of  the  target.  The  main  goal  is  to 

estimate  n  by  employing  an  imperfect  sensor  to  look  into  the  area-cells.  The  sensor  is 
characterized  by  its  sensitivity  and  specificity.  For  each  area-cell, 

aj  =  /’(sensor  indicates  detection  in  area  cell  i  |  target  is  in  area-cell  /  ) 
is  the  sensitivity,  and  1  -bj  is  the  specificity,  where 

=  P (sensor  indicates  detection  in  area  cell  i  |  target  is  not  in  area-cell  /  ). 

We  assume  that  the  sensitivity  and  specificity  are  known.  Suppose  at  >  b:  (otherwise  we 
can  reverse  the  sensor  cue,  meaning  that  a  “target  present”  indication  is  interpreted  as 
“target  absent,”  and  vice  versa). 

Consider  area-cell  i.  Let  Xn,Xj2,...  be  independent  and  identically  distributed 
(IID)  random  variables  that  describe  the  sensor  observations,  where  X.  = 1  if  the  sensor 
returns  a  detection  (hot)  signal  in  the  j '  th  look  into  area-cell  i,  and  W  =0  if  the  sensor 

returns  a  no  detection  (cold)  signal.  Thus  (AW  j  ^  is  a  collection  of  Bernoulli  IID 

random  variables,  with  P{X.]=\}  =  (\  —  7ii')bi  +7tiar  We  defer  to  further  study  the 

option  to  relax  the  IID  assumption  in  order  to  allow  for  some  correlation  among  sensor 
observations. 
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The  decision  variables  are  pl,...,pm,  the  fraction  of  the  search  budget  allocated  to 

each  area-cell,  where  ^  p.=  1  and  pi>  0.  The  search  budget  is  described  by  n.  In  this 

work,  we  are  interested  in  determining  efficient  search  allocations  for  n  large.  It  is 
therefore  reasonable  to  assume  that  the  TOI  is  already  in  steady-state. 


2.  Large  Search  Budget  Results 


Let  Xi(pjn)  be  the  fraction  of  detections  in  area-cell  i  by  the  time  of  the 

[ p  ti\  ’th  look  (in  what  follows  we  work  with  ptn  instead  of  [ p.ti\ ;  since  our  results 

hold  for  n  large,  they  continue  to  be  true  when  the  integrality  condition  is  enforced,  by 
working  with  a  sequence  that  goes  to  infinity).  By  the  Strong  Law  of  Large  Numbers,  we 
know  that 


XSP,n)-b! 

j  JLi 

a,-bt 

with  probability  1  as  n  — »oo  (Ross,  1996,  p.  41). 

We  choose  to  minimize  the  largest  absolute  error.  Thus,  by  standard  results  in 
large  deviations  theory  (Dembo  and  Zeitouni,  1998)  we  have 


X,(p,n)-bi 


-  —  71; 


>  £ 


ai~hi 

=  +  b,  (1  - ni ))|  >  (a,  -bt ) s) 


(1.1) 


~  exp (~Pin  min {/,.  (bt  +  (a,.  - b, ) (tt,  +  e)) ,  /  (b  +  (a. -b,  £'))[), 

where/,. (•)  is  the  large  deviations  rate  function  (see  Dembo  and  Zeitouni,  1998,  p.  4). 
Also, 


ma  xP 


X,(pin)-bi 


a ■  -  b- 


-  —  71; 


> 

f 

>  £ 

<P 

max 

J 

l  '■ 

Xi{pin)-bj 


a .  —b: 


-71; 


>  £ 


<  m  max  P 


Xl(p,n)-bi 


CL;  ~b; 


-—71; 


(1.2) 


>  £ 


Combining  the  outcomes  of  (1.1)  and  (1.2)  results  in 
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(1.3) 

->  -min p,  min}/,.  (bi+(ai  -&,)(*,+*)),/,  (bt  +  (a,.  -£,/)(*,  -*))}, 

as  n  — >  oo. 

For  a  non-degenerate  Bernoulli  random  variable  Xn  with  mean  the 

large  deviations  rate  function  /,.  is  given  by 

f  A  (■.  _  \ 

Ii(rt)  =  rAo. g  —  +(!-/,■) log  — -  , 

V  P>  J  V1  Mj 

for  0  <  y.  <  1 . 

The  optimal  allocation  solves 

max  min,.  Pi  min j/,.  (/,  +(a,  -h,  )(^, +^)),/,  (h,+(a,  "*))} 

s.t. 

m 

Z  a  - 1 

1=1 

Pi>0 

and  makes  equal  the  exponential  decay  rates  that  appear  in  the  right  hand  side  of 
Equation  (1.1),  for  each  area-cell  The  (unique)  solution  is  given  by 

min ili(bi+{ai-bi)(Ki  +  s)),Ii(bi+{ai-bi)(jci-e))}  ' 

Pi  = - - - - - -j- , 

ZJ=imin{7y  +  (a,  -^)(^  +^)),//(^  +(fly -*,)(*;-*))} 

where  we  choose  £  sufficiently  small  so  that  the  argument  inside  each  logarithm  is  non¬ 
negative.  The  steady-state  distribution  n  is  unknown,  so  it  is  replaced  by  the  standard, 
sampling-based  estimator  that  is  obtained  as  the  stochastic  approximation  algorithm 
(described  in  the  next  section)  makes  progress. 


—  log/5  max 
n  ' 


Xi(Pjn)-bi 
a,  -  b: 


-n\>  s 
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3. 


Stochastic  Approximation  Algorithm 


We  first  present  the  algorithm,  with  the  intuition  behind  it  following  immediately 
thereafter. 

Initialization.  Let  0  <  X  0  <  1  be  our  initial  guess  of  bi  +  (ai  -bi)ni ,  and  set 
=Xj0  +  (cr  -bi )  s,  and  y'.  0  =  W  0  -  (a,.  -bt )  s.  The  initial  guesses  of  the  rate  functions 


are 


Ko  =  tio  lo§ 


and 


Ko  =  Yifi  lo§ 


+  (1-n»)log 


,M4 


(i-Lo) 


The  initial  guess  of  the  optimal  allocation  is 


mm 


Pi,  0  = 


Kc.-Of 


Finally,  set  £  =  0  . 

Algorithm  SA. 

1.  Generate  a  replicate  £  from  the  probability  mass  function  pxl,...,pmt. 

2.  Update  sample  sizes:  U  M  =  U  ,  + 1,  and  Xt  M  =  X: ,  for  i  ^ 

3.  Generate  a  sample  from  area-cell  g,  (say)  AT  x  ,  from  a  Bernoulli  with  parameter 
a, 7i,  +b^{\- 7i\ 

4.  Update  Xt^e+l,y^e+l,y^M,I^M,  and ■ 


X 


Set 


rf. 


i+ 1 


=  xs  + 


and  y{J+l 


=  V( 


a.  Let 
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To  see  why  our  algorithm  leads  to  the  optimal  allocations,  let  9t ,  =  Xut  II  be  the 
fractional  allocations  in  stage  l  of  the  algorithm.  Hence,  step  2  of  the  algorithm  can  be 
expressed  as  9i  M ,  =  9j ,  +  ( J  ( <g,  =  /)  ~  0j  ,)  /  ( t  + 1) ,  where  q,  is  the  £th  replicate  of  q 

generated  in  step  1  of  the  algorithm,  and  /(•)  is  the  indicator  function.  The  recursion  for 
9i  M  can  be  re-written  as 

&ue+ i  =  &i,e  +  J~[  ( Pi  ~  )  +  £f’ 

where 

=  ^  '  Pu ) “ p‘  )■ ■ 

If  the  error  s,  becomes  small  relative  to  the  ( p*  -0.  ( + 1)  term,  then  9t ,  follows,  as 
f  — >  oo,  the  path  of  the  solution  of  the  ordinary  differential  equations 

9'i=p]~9i,  i  = 

which  have  p*  as  the  unique  globally  asymptotically  stable  point.  This  suggests  that  if 
the  variability  introduced  by  the  error  at  each  stage  is  sufficiently  small 
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(i.e.,  var(£-j)  +  ...  + var^)  =  o(f)),  our  algorithm  provides  fractional  allocations  that 

converge  almost  surely  to  the  optimal  allocations.  A  rigorous  analysis  of  this  approach  is 
discussed  in  Kushner  and  Yin  (2003,  p.  170). 

B.  SST  MODEL 


1.  Case  1:  Independent  Area-cells  with  No  Target  or  One  Target 


While  not  an  operationally  relevant  or  likely  scenario,  we  treat  the  case  where 
each  area-cell  either  contains  or  does  not  contain  a  TOI,  independent  of  all  other  area- 
cells,  because  it  brings  intuition  about  the  single-target  cases  (case  2  and  case  3)  of  this 
chapter. 

Procedure  SPl 


Consider  one  area-cell,  and  suppose  that  the  sensor  operator  prescribes  an 
acceptable  false-positive  probability  a  and  a  false-negative  probability  /? .  The 

observations  X1,X2,...  are  drawn  from  a  Bernoulli  random  variable  with  parameter  a  if 
the  area-cell  contains  a  target,  or  from  a  Bernoulli  random  variable  with  parameter  b  if 
the  area-cell  does  not  contain  a  target.  Let  Sn  =  Xl  +  ...  +  Xn  be  the  number  of  detections 

after  n  looks.  For0</?<1,  the  likelihood  function  ps"  (1-/?)'  S"  can  be  used  to 

determine  whether  the  unknown  Bernoulli  parameter  is  a  or  b  ,  because  this  likelihood  is 
maximized  by  p  =  a  if  the  area-cell  contains  a  target  and  by  p  =  b  if  it  does  not.  Hence 
the  likelihood  ratio 


as"(l-a)n 


•  00 


bs"(l-by~s" 

if  a  target  is  present,  and  £n  (jc1?.  ..,xn )  — »  0  if  the  area-cell  is  target-free,  as  n  — »  co  .  This 


suggests  that  a  judicious  policy  is  to  stop  sampling  when  the  likelihood  ratio  crosses  an 
upper  threshold  and  declare  the  target  present,  or  when  the  likelihood  ratio  crosses  a 
lower  threshold  and  declare  the  target  absent.  This  approach  may  lead  to  an  incorrect 
determination,  but  its  probability  can  be  prescribed  ab  initio  by  the  end-user. 
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Define  the  hypotheses: 

74  :  Target  absent  from  area-cell,  and  7T,  :  Target  present  in  area-cell. 

Given  our  definitions  of  sensor  sensitivity  and  specificity,  these  hypotheses  are  analogous 
to  sampling  from  a  Bernoulli  distribution  with  parameter  p  where 

74  :  p  =  b,  or  7 \\p  =  a. 

Define  the  stopping  time 

TV  =  inf  jn  >  1 :  f  n 

for  the  threshold  constants  A,  B  such  that  -00  <  A<  B  <00.  Then  we 
Reject  7-4  if  t  N  >  B  and  Accept  7 4  if  l  N  <  A. 

For  0  <  a,  /?  <  1  prescribed  by  the  end-user,  the  error  probabilities  are 

Type  I  error:  P(tN  >  B  |  74)  =  oc  and  Type  II  error:  P(£  N  >  A  |  7f,)  =  J3 . 

Taking  logarithms,  we  can  see  that  we  reject  74  if 


log  B 


log 


log 


a  1  -b 
b  \-a ) 


+  N- 


1  -b 
1  -a 


log 


a  1  -b 
b  1  -a  ) 


V 


(We  know  the  denominator  is  positive;  recall  our  assumption  that  a  >  b,  otherwise  we 
can  reverse  the  sensor  cue)  We  accept  74  if 


log  A 


log 


r a  1 -b^ 
Kb  1  -a  j 


+  N- 


1  -b 
1  -a 


log 


log 


a  1  -b 
\b  1  -a ) 


Y 


Siegmund  (1985,  p.  10)  shows  that 

and 

P  =  Pl(t„<A)<A(\-a). 

Hence,  given  operator  defined  tolerances  a  and  /?,  by  setting 

5  =  -^,  and  A  =  ^ 


a 


1  -a 


(1-4) 

(1-5) 
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we  are  guaranteed  to  satisfy  the  error  probability  constraints.  Moreover,  it  is  shown  in 
Siegmund  (1985,  p.  11)  that  if  Equations  (1.4)  and  (1.5)  hold  with  equality,  then  this 
approach  minimizes  the  expected  number  of  looks  until  crossing  either  boundary. 
Although  (1.4)  and  (1.5)  generally  do  not  hold  with  equality,  the  algorithm  is  guaranteed 
to  meet  the  error  criteria 

Figure  3  displays  two  possible  sample  paths  computed  by  SP 1.  In  this 
illustration,  the  white  area  represents  the  state  space  for  the  total  number  of  detections 
after  n  looks  (Sn ) ,  and  the  two  parallel  dashed  lines  represent  the  bounds.  A  path 

between  the  bounds  is  still  undetermined;  thus  the  procedure  continues  until  the  path 
exits  via  one  of  the  bounds  (hence,  this  area  is  known  as  the  continuation  region).  A 
sample  path  that  exits  the  upper  bound  results  in  a  declaration  of  target  present  in  the 
area-cell,  whereas  a  path  exiting  via  the  lower  bound  produces  a  target  absent 
declaration. 
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number  of  looks 

Figure  3.  Example  sample  paths  of  the  one-cell  sequential  procedure. 

Next  we  discuss  the  case  of  a  single  target  hidden  among  m  area-cells,  with 
m  >  1 .  This  case  is  difficult  to  analyze  because  knowledge  about  the  presence/absence  of 
a  target  in  an  area-cell  yields  light  about  the  presence/absence  of  the  target  in  other  area- 
cells;  i.e.,  the  declarations  about  target  absence/presence  in  each  area-cell  are  no  longer 
independent.  In  order  to  set  the  stage,  we  start  with  the  case  m  =  2. 

2.  Case  2:  Single  Target  and  Two  Area-cells 

Procedure  SP2 

For  this  slightly  more  complicated  case,  consider  (as  a  starting  point)  two  area- 
cells,  AC0  and  AC] ,  with  parameters  ( a0 , b0 )  and  (a,,^),  respectively,  such  that 
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ai>bj  for  7  =  0,1.  For  simplicity,  we  assume  that  both  area-cells  receive  the  same 

number  of  looks,  and  the  goal  is  to  terminate  the  inspection  when  there  is  enough 
evidence  that  the  error  bounds  are  met,  i.e.,  we  are  confident  to  within  our  error 
tolerances  of  saying  that  the  TOI  is  in  a  particular  area-cell.  The  two  possible  errors  are: 

(i)  a  is  the  probability  that  the  target  is  determined  to  be  in  A  C]  when  it  is  in  A  C0 ,  and 

(ii)  [i  is  the  probability  that  the  target  is  determined  to  be  in  AC0  when  in  reality  it  is  in 
ACy  This  leads  to  the  hypotheses 

Hq  :  Target  located  in  AC0,  and  7 T, :  Target  located  in  ACr 
Let  Sin  be  the  number  of  detections  in  A C) ,  i  =  0,1,  after  n  looks.  Consider  the  ratio  of 
likelihood  ratios  £n  (often  referred  to  as  the  “odds  ratio”), 


where 


We  know  that 


•  oo 


a,'" 

if  the  target  is  present  in  A  C]  and 


•  0  if  it  is  absent.  Hence 


•  oo 


in  — >  0  if  the  target  is  present  in  A  C0 ,  and  i 
the  stopping  time 

N  =  inf{«  >\\in  £  (A,B)} 

for  threshold  constants  A,  B  such  that  -oo  <  A  <  B  <  oo.  Then  we 
Reject  Hq  if  tN  >  B,  and  Accept  77J,  if  lN<  A. 
Taking  logarithms,  we  can  see  that  we  reject  /Hi)  if 


otherwise.  This  suggests  considering 


ShN  log 


f  ax  1-hj  ^ 


^bx  \-ax 
and  we  accept  7 -4  if 


~So,N  log 


SLN  log 


^  a,  1  —  bx  A 
ybx  1  -axJ 


~S0 ,n  log 


f  a0  1-0 

V^o  1 —  ao  J 


f  a0  1-fcp^ 

\b0  l  —  a0j 


+  A  log 


f  \  -ax  1  -b0  ^ 


1-hj  1  -  a 


>logR, 


+  A  log 


0  J 


f  \  -ax  1  -bt  ^ 


1  -bx  l-o 


<  log  A. 


0  J 


(1.6) 
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Siegmund  (1985)  shows  that 


a  =  P„(lM>B)<B-'(\-l3) 

(1.7) 

and 

P  =  Pl(lH<A)<A(\-a). 

(1.8) 

Hence,  given  operator  defined  tolerances  a  and  /?,  by  setting 

B  =  - — — ,  and  A  =  ^  , 
a  1  -  a 

(1.9) 

we  are  guaranteed  to  satisfy  the  error  probability  constraints. 
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Figure  4  shows  two  possible  sample  paths  for  SP2.  Again,  the  continuation 
region  is  the  area  between  the  two  dashed  lines  representing  the  bounds.  A  path  exiting 
the  bound  corresponding  to  log  (B)  indicates  a  belief  of  target  presence  in  ACV  whereas 

an  exit  via  the  log^)  bound  indicates  belief  of  target  presence  in  AC0. 
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Figure  4.  Example  sample  paths  for  the  two-cell  SST model  case. 


3.  Case  3:  Single  Target  in  One  of  m> 2  Area-cells 
Procedure  SP3 

Consider  an  AOI  consisting  of  m  >  2  area-cells,  and  a  single  TOI  located  within 
the  AOI.  Assume,  without  loss  of  generality,  that  the  target  is  hidden  in  ACr  The 
operator  can  specify  the  error  tolerance  in  many  ways  in  this  case;  for  example,  “If  the 
TOI  is  in  ACr  I  want  the  probability  of  saying  that  the  TOI  is  in  AC2  to  be  less  than  3%, 
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and  the  probability  of  saying  the  TOI  is  in  AC3  to  be  less  than  2%,”  etc.  For  simplicity, 
in  this  thesis,  consider  that  the  only  possible  error  in  this  situation  is  to  conclude  that  the 
target  is  not  in  A C, .  We  call  such  event  ICD  (for  incorrect  determination ),  and  let 

f5(/CD)  <  a  ,  for  some  a  e  (0, 1)  pre-specified  by  the  operator. 

Let  C  be  the  set  of  candidate  area-cells;  initially  all  ACs  are  candidates  to 
contain  the  target,  so  that  C  =  m}  .  Let  be  the  largest  likelihood  ratio  at  stage 

n .  The  idea  is  to  sequentially  drop  an  area-cell  from  consideration  when  there  is 
sufficient  evidence  that  it  does  not  contain  the  target,  i.e.,  when  we  are  confident  to 
within  our  error  tolerance  of  saying  that  the  TOI  is  not  in  that  particular  area-cell.  This 
suggests  eliminating  ACi  when  t{V)  n  / £in  >B,  where  B  is  selected  to  satisfy  the  bound 

P(lCD)  <  a. 


To  be  more  precise,  consider  area-cell  1  (which  contains  the  TOI)  and  an  arbitrary 
area-cell  /  Wl .  Given  thresholds  0  <  A  <  B ,  let  Nt  =  inf  jn  >  1 :  i . n  it t n  <£  (A,B) }  be  the 
first  time  the  odds  ratio  of  area-cells  1  and  i  exits  the  interval  ( A,B ).  Following 
Equations  (1.7)  through  (1.9),  with  B  =  A~l  =  (m-X)/ a  - 1  ,we  can  guarantee  the  error 
bound  P  (f  ( Nj  /  i  j  Nj  >  B}<  a  /  (m  - 1)  .  By  Bonferroni’s  inequality,  it  follows  that 

(\  m  m  r/ 

u; >5  <£/>(«,,»,/«,.»,  ^5)<£ - -  =  a. 

'  1=2  i' =2  m  t 

Rather  than  pair-wise  comparing  all  area-cells,  it  suffices  to  drop  from  consideration  any 
area-cell  i  for  which  lf]]  n  /  n  >  (m  - 1)  /  a  - 1 . 


The  algorithm  proceeds  as  follows: 

Algorithm  SE 

1.  Obtain  one  signal  {sample)  from  all  area-cells  i  e  C. 

2.  Compute  n+1  and  the  ratios  f(|)  n+1  /  i.  n+1 ,  Vz  e  C. 

3.  If  n+l  /  £.  n+1  >  (m  - 1)  /  a  - 1,  then  remove  i  from  C. 
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4.  If  C  =  1,  stop  and  declare  the  single  AC  in  C  the  determined  area-cell. 
Otherwise,  increase  n  <—  n  + 1,  and  go  back  to  1 . 

The  SE  algorithm  is  guaranteed  to  meet  the  operator-defined  tolerance  a  for  the 
probability  of  incorrect  determination. 
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IV.  COMPUTATIONAL  STUDY 


This  chapter  presents  results  obtained  by  numerical  experimentation  in  MATLAB. 
We  begin  our  discussion  with  a  description  of  a  notional  operational  scenario  to  frame  the 
SMT  model.  We  then  explain  the  implementation  of  the  stochastic  approximation 
algorithm  presented  in  Chapter  III  in  the  context  of  this  scenario.  Finally,  we  discuss  the 
results  of  the  numerical  experiments,  and  present  an  analysis  of  stochastic  approximation 
algorithm  performance.  We  then  repeat  the  process  for  the  case  of  the  SST  model  and 
sequential  eliminating  procedure. 

A.  SMT  MODEL 

1.  Scenario  Development 

For  the  purposes  of  our  study,  we  place  our  notional  area  of  operational  interest 
(AOI)  in  a  39  square  kilometer  section  of  downtown  Baghdad,  Iraq.  Although  the  SMT 
model  does  not  require  that  area-cells  be  uniform  (in  size  or  shape)  or  geographically 
adjacent,  we  partition  our  AOI  in  this  manner  as  a  matter  of  illustrative  and 
computational  convenience.  Thus,  we  begin  by  discretizing  the  AOI  into  uniform  area- 
cells  of  size  500  by  500  meters.  Figure  5  depicts  the  resulting  AOI,  consisting  of  156 
area-cells  in  a  13  by  12  rectangular  grid. 
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Figure  5.  Discretized  notional  scenario  Area  of  Operational  Interest. 
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At  the  beginning  of  the  scenario,  a  single,  randomly  moving  high  value  target  of 
interest  (TOI)  is  located  somewhere  within  the  AOI.  In  our  case,  suppose  the  TOI  is  a 
medium-sized  white  sport  utility  vehicle  known  to  be  an  insurgent  weapons  cache.  An 
intelligence  agency  has  recorded  the  TOI’s  hyperspectral  signature  and  provided  the 
library  data  to  the  searcher.  We  construct  the  initial  TOI  probability  distribution  by 
starting  with  a  Threat  Map  (Riese,  2006),  which  we  aggregate  and  normalize  to  produce  a 
probability  map  that  is  compatible  with  our  AOI,  and  whose  probabilities  sum  to  one. 
Table  1  summarizes  the  initial  probability  map,  with  probabilities  to  three  decimal  places. 

We  assume  the  TOI  moves  in  a  Markovian  fashion,  with  a  transition  matrix  that  is 
unknown  to  the  searcher.  Appendix  C  shows  the  non-zero  columns  of  the  ground-truth 
transition  matrix  used  in  our  scenario.  Table  2  depicts  the  true  steady-state  distribution  of 
TOI  location  (again,  unknown  to  the  searcher)  resulting  from  this  transition  matrix,  to 
three  decimal  places.  For  purposes  of  the  numerical  experiment,  we  assume  that  the 
target  has  already  reached  steady-state. 

The  searcher  possesses  a  UAS-bome  hyperspectral  sensor,  and  a  certain  number 
t  of  available  looks,  or  search  budget.  In  order  to  preserve  this  thesis  as  unclassified,  we 
assign  reasonable  random  values  of  hyperspectral  sensor  sensitivity  and  specificity 
against  a  known  signature.  Specifically,  we  assign  a  random  uniform  value  between  0.75 
and  0.99  for  sensitivity  and  a  random  uniform  value  between  0.89  and  0.99  for  specificity 
to  each  area-cell,  as  depicted  to  two  decimal  places  in  Table  3.  It  is  important  to  note 
that,  despite  our  somewhat  cavalier  method  of  assigning  these  numbers,  detennining  the 
appropriate  values  operationally  can  be  highly  complex  (as  discussed  in  Chapter  II),  and 
is  outside  the  scope  of  this  thesis.  It  is  left  as  an  option  for  further  study  to  account  for 
noise  in  the  measured  values  of  sensitivity  and  specificity. 
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Table  1.  Initial  TOI  probability  map,  derived  by  normalizing  the  aggregate  Threat  Map 

(After  Riese,  2008). 
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Table  2.  Notional  TOI  steady-state  distribution. 
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0.894 
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O  0.785 
9  0.985 

Table  3.  Notional  hyperspectral  sensor  sensitivity  (center  cell  values)  and  specificity 

(bottom  cell  values). 
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In  each  time  step,  the  searcher  allocates  one  look  in  a  chosen  area-cell,  and  the 
search  budget  is  subsequently  decremented.  Upon  each  look,  the  sensor  returns  a  signal 
of  hot  if  it  senses  detection  (of  course,  correct  detections  and  false  positives  are 
indistinguishable  to  the  sensor),  and  a  signal  of  cold  otherwise.  The  scenario  then 
advances  forward  one  time  step,  and  the  process  continues  until  the  entire  search  budget 
is  exhausted.  Recall  that  the  objective  for  the  searcher  is  to  determine  the  steady-state 
location  distribution  of  the  TOI  as  quickly  as  possible  (to  within  the  absolute  error 
tolerance),  based  on  the  signals  from  the  sensor  and  knowledge  of  its  sensitivity  and 
specificity. 

2.  Stochastic  Approximation  Algorithm  Implementation  and  Results 

We  implement  our  model  in  the  framework  of  the  described  notional  operational 
scenario,  using  MATLAB.  Sample  MATLAB  code  for  the  SA  algorithm  may  be  found  in 
Appendix  A. 

In  order  to  evaluate  the  SA  algorithm’s  ability  to  approximate  the  true  steady-state 
distribution  of  the  target  and  the  associated  near-optimal  sampling  rates,  one  replication 
of  50,000  iterations  was  performed  on  the  156-cell  AOI.  Table  4  depicts  the  algorithm’s 
cell  sampling  rates  compared  to  the  theoretical  optimal  area-cell  sampling  rates  which,  as 
discussed  in  Chapter  III,  depend  not  only  on  the  absolute  error  tolerance  (c),  sensor 

sensitivity  (a.),  and  specificity  ( 1  -  hi )  values,  but  also  on  the  true  steady-state 
distribution  Cell  sampling  rates  are  given  to  four  decimal  places. 
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Table  4.  Theoretical  optimal  search  frequencies  (center  cell  values)  and  SA  algorithm 
resultant  search  frequencies  (bottom  cell  values,  based  on  50,000  iterations). 
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It  is  difficult  to  infer  a  degree  of  model  success  based  upon  these  values,  beyond 
the  fact  that  the  SA  algorithm  provides  sampling  rates  that  appear  to  be  on  the  same  order 
of  magnitude  as  the  theoretical  optimal  sampling  rates.  It  is  therefore  more  useful  to 
evaluate  the  model  based  upon  its  ability  to  estimate  the  TOI’s  steady-state  distribution  to 
within  a  certain  absolute  error  tolerance,  which  we  provide  below  in  Section  3. 

Figure  6  depicts  estimates  of  the  true  steady-state  target  position  distributions  for 
three  select  area-cells.  The  estimates  provided  by  the  SA  algorithm  are  compared  to  the 
estimates  generated  by  a  unifonn  random  search.  Both  estimates  appear  to  converge 
upon  the  true  steady-state  distribution  as  the  total  number  of  looks  becomes  large,  in 
accordance  with  the  Strong  Law  of  Large  Numbers.  One  might  conjecture  from  this 
figure  that  the  total  error — depicted  by  the  aggregate  area  between  each  estimate  and  the 
true  steady  state  after  the  transient  has  worn  off — is  less  for  the  SA  estimate  than  for  the 
random  unifonn.  However,  it  is  difficult  to  say  whether  this  is  in  fact  the  case,  or  just  an 
artifact  of  this  particular  numerical  experiment,  as  this  is  only  the  figure  only  shows  one 
replication’s  data  for  three  area-cells.  This  further  suggests  that  a  more  useful  measure  of 
model  effectiveness  will  be  the  error  decay  rate,  again  provided  in  Section  3. 
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Figure  6.  Number  of  looks  versus  steady-state  estimate  for  one  sample  path  of  three 

particular  area-cells. 

In  order  to  evaluate  the  absolute  error  rate,  we  count  the  number  of  replications 
for  which  the  algorithm’s  estimate  of  the  steady-state  distribution  is  outside  of  some  pre¬ 
defined  absolute  tolerance  (in  our  case,  an  absolute  error  tolerance  of  0.05  was  used),  for 
search  budgets  at  each  integer  level  between  1  and  6500,  and  divide  by  the  total  number 
of  replications.  Due  physical  memory  and  time  constraints  (a  single  replication  on  the 
156-cell  AOI  took  over  two  hours  to  compute),  the  absolute  error  decay  rate  was 
evaluated  by  conducting  10,000  replications  with  a  simplified  AOI  of  only  four  area-cells 
(results  are  depicted  in  Figures  7  and  8).  However,  the  error  rate  results  generalize  to 
problems  of  any  size,  as  shown  in  Chapter  III. 

3.  Analysis  of  Stochastic  Approximation  Algorithm  Performance 

Recall  that  the  measure  of  effectiveness  in  this  model  is  to  minimize  the 
probability  that  the  resultant  absolute  error  in  the  estimate  of  steady-state  distribution  is 
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greater  than  some  tolerance  a .  In  Chapter  III,  we  acknowledged  that  any  search  plan 
that  allocates  a  positive  fraction  of  the  search  budget  to  all  cells  would  lead  to  an  absolute 
error  rate  that  decays  to  zero  exponentially  as  the  search  budget  goes  to  infinity.  Figure  6 
supported  this  by  showing  that  the  SA  algorithm  estimate  and  a  random  uniform  estimate 
both  converge  to  the  true  steady-state  values. 

Additionally,  we  claimed  that  the  search  frequencies  detennined  by  the  SA 
algorithm  exhibit  error  rates  that  decay  at  the  fastest  rate  possible.  Figure  7  depicts  the 
average  resultant  absolute  error  decay  rates  (over  10,000  sample  paths  of  6,500  iterations 
on  a  simpler,  four-cell  problem)  for  four  different  strategies: 


•  A  random  unifonn  search. 


•  A  constant  naive  estimate,  where  the  determination  thresholds  are  the 
midpoints  between  a.  and  bi.  If  the  allocations  are  inversely  proportional  to 

the  distance  a,  — —  ,  the  allocation  becomes  p;  =  — — - — - T  . 

2 

•  The  stochastic  approximation  algorithm  search  discussed  in  this  section. 


•  A  hypothetical  optimal  search  plan  in  which  the  searcher  knows  a  priori  the 
theoretical  optimal  search  frequencies  derived  in  Chapter  III. 
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Figure  7.  Large  number  of  looks  versus  the  logarithm  of  probability  of  error. 

As  predicted,  the  a  priori  plan  appears  to  provide  a  bound  on  error  decay  rate  and 
as  the  search  budget  grows  large,  the  SA  algorithm  outperforms  the  random  uniform 
search.  The  naive  allocation  performs  relatively  well,  with  an  error  rate  tending  more 
toward  the  optimal  than  toward  the  random  (see  Figure  7). 

One  important  feature  of  the  model  deserves  discussion.  Recall  that  a  limitation 
of  the  SMT  model  is  that  it  applies  only  to  situations  involving  large  search  budgets. 
Note  that,  for  a  relatively  small  search  budget  (say,  less  than  about  1000  in  the  current 
example),  any  efficiency  gained  by  the  SA  algorithm  is  negligible.  In  fact,  it  is  often  the 
case  that  the  random  uniform  error  rate  decays  faster  for  than  that  of  the  SA  algorithm 
when  the  number  of  looks  is  small  (see  Figure  8).  Additionally,  the  naive  allocation 
performs  nearly  as  efficiently  as  the  random  in  this  situation. 


46 


0 


T 


T 


-0.2 


-0.4 


_  -0.6 
l— 

2 
i— 

<D 

o  -0.8 


-Q 

5  -1 

2 
Q. 

n> 

5  -1.2 


-1.4 


-1.6 


-1.8 


\ 


random  uniform 


stoch  approx 


>< 


naive 


a  priori 


0  100  200  300  400  500  600  700  800  900  1000 

number  of  looks 


Figure  8.  Small  number  of  looks  versus  the  logarithm  of  probability  of  error. 


B. 


SST  MODEL 


1.  Scenario  Development 

Consider  the  same  discretized  AOI  as  for  the  adaptive  case  (see  Figure  5).  Sensor 
sensitivity  and  specificity  remain  as  in  Table  3.  We  first  consider  the  case  where  each 
area-cell  either  contains  or  does  not  contain  a  stationary  TOI,  independent  of  all  other 
cells,  and  refer  to  this  as  case  1 .  This  allows  us  to  perform  the  calculations  on  one  area¬ 
cell,  and  generalize  the  results  to  all  area-cells.  We  then  modify  the  scenario  to  the  case 
in  which  a  single  stationary  TOI  is  present  in  only  one  of  two  area-cells  (case  2).  Finally, 
we  generalize  to  the  case  of  m  >  2  area-cells  (case  3). 
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2.  Sequential  Eliminating  Procedure  Implementation  and  Results 


As  with  the  SMT  model,  we  choose  MATLAB  as  a  computational  tool  for  the 
numerical  experiments  for  the  SST  model.  Sample  MATLAB  code  for  the  sequential 
eliminating  procedure  may  be  found  in  Appendix  B. 

For  case  1,  an  arbitrary  area-cell  containing  a  TOI  was  chosen.  To  observe  the 
effect  of  decreasing  the  difference  between  sensor  sensitivity  and  (1 -specificity)  on  the 
expected  number  of  looks,  b  was  held  constant  at  0.35,  and  a  was  varied  from  0.7  to 
0.35  at  increments  of  0.002.  10,000  replications  were  conducted  at  each  increment. 

Predictably,  the  closer  the  values  of  a  (sensitivity)  and  b  (1 -specificity)  are  to  one 
another,  the  larger  the  number  of  looks  required  to  make  a  determination.  Figure  9 
verifies  this  intuition;  in  fact,  the  increase  appears  to  be  polynomial  in  the  complement  of 
the  difference. 


O 
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Figure  9.  Polynomial  increase  in  number  of  looks  as  difference  between  a  and  b  gets 

small  (m  =  1,  a  =  0.05). 
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For  cases  2  and  3,  we  arbitrarily  (and  without  loss  of  generality)  placed  the  target 
in  A Cx .  The  error  tolerance  a  remained  fixed  at  0.05.  Starting  with  m  =  2  area-cells 

and  working  up  by  adding  one  AC  per  iteration  until  encompassing  all  156  area-cells, 
we  performed  50,000  replications  at  each  iteration  in  order  to  determine  achieved  error 
rates  and  expected  number  of  looks  as  m  increases.  Figure  10  depicts  the  near-linear 
increase  in  observed  number  of  looks  as  m  increases  for  case  3  with  fixed  a  =  0.05. 
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Figure  10.  Near-linear  relationship  between  m  and  number  of  looks,  case  3,  a  =  0.05. 

To  determine  the  effect  of  varying  error  tolerance,  we  chose  to  fix  m  =  10 .  The 
type-I  error  probability  threshold  a  was  varied  between  0.01  and  0.1  at  0.001 
increments.  At  each  level  of  a ,  50,000  replications  were  perfonned  to  calculate  the 
observed  miss  rate  and  average  number  of  looks  until  a  detennination  of  target  presence 
or  absence  was  made.  Naturally,  the  number  of  expected  looks  decreases  with  an 
increase  in  error  tolerance  for  the  sequential  procedure.  Figure  1 1  highlights  the 
relationship. 


49 


alpha 

Figure  1 1 .  Effect  of  varying  a  on  expected  number  of  looks  (m=\  0). 


3.  Analysis  of  Sequential  Eliminating  Procedure  Performance 

Recall  that  the  goal  of  the  sequential  eliminating  procedure  is  to  indicate  target 
location  with  an  accuracy  rate  guaranteed  to  meet  operator-specified  error  tolerances, 
within  a  reasonable  number  of  expected  looks.  One  possible  measure  of  performance  is 
the  amount  of  slack  between  the  error  tolerance  and  the  observed  error  rate.  Intuitively, 
the  less  slack,  the  fewer  number  of  expected  looks  would  be  required.  However,  if  a 
method  with  the  same  expected  number  of  looks  as  the  sequential  eliminating  procedure 
exhibits  a  larger  observed  error  rate,  it  is  reasonable  to  state  that  the  sequential 
eliminating  procedure  is  more  efficient  than  such  a  method.  (Alternatively,  one  could 
invoke  a  method  exhibiting  the  same  achieved  error  rate  and  compare  the  expected 
number  of  looks,  but  we  choose  the  former  scheme  for  ease  of  computation  and 
illustration.) 

For  comparison  purposes,  consider  a  naive  allocation  method  with  the  same  total 

expected  number  of  looks  as  the  sequential  eliminating  procedure.  Let  fn  be  the  total 
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average  number  of  looks  for  the  sequential  eliminating  procedure  with  an  AOI  consisting 
of  m  area-cells.  Let  Xm  be  the  average  number  of  looks  per  area-cell  for  the  naive 
allocation,  which  allocates  an  equal  number  of  looks  to  each  area-cell,  and  does  not 
eliminate  area-cells  from  contention,  so  that  Am  =  fm  /  m.  (Since  Am  is  likely  not  an 
integer  value,  we  set 

1X1  w-p  -p 

Am  = < 

JXJ  w-p-  l-p> 

where  p  =  Xm  -  \^Am  J ,  in  order  to  approach  the  desired  average  over  many  iterations.) 
After  Xm  looks  into  each  area-cell,  the  area-cell  with  the  largest  likelihood  ratio  is 

declared  to  contain  the  target  for  the  naive  model.  We  begin  our  comparison  by 
examining  the  effect  of  varying  a  with  m= 10.  We  then  fix  a  and  complete  5,000 
iterations  at  each  integer  m  =  3,...,  156,  comparing  the  observed  error  rates  of  the 
sequential  eliminating  procedure  with  that  of  the  naive  model. 

Figure  12  shows  that,  with  fixed  a,  b  and  ft  for  the  multiple-cell  case,  an 
increase  in  a  appears  to  affect  an  increase  in  the  observed  miss  rate  in  a  manner  that 
preserves  the  ratio  of  a  and  the  observed  rate  for  the  sequential  eliminating  procedure. 
Also  note  that,  for  the  same  average  number  of  looks  at  each  value  of  a,  the  observed 
error  rate  for  the  naive  method  is  considerably  larger  for  all  values  of  a  (indeed,  the 
naive  rates  exceed  the  threshold  for  all  values  of  a  ). 
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Figure  12.  Effect  of  varying  a  on  observed  error  rates  (m= 10). 

Figure  13  illustrates  how  the  observed  error  rate  for  the  sequential  eliminating 
procedure  appears  to  converge  as  the  number  of  area-cells  becomes  large.  The  observed 
rate  is  arguably  reasonable  when  compared  to  the  threshold.  As  with  the  case  of  varying 
a,  the  error  rate  for  the  naive  method  is  considerably  higher  than  that  of  the  sequential 
eliminating  procedure  for  the  same  total  number  of  looks  for  all  values  of  m.  Indeed,  the 
naive  error  rate  is  once  again  consistently  above  the  threshold. 
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Figure  13.  Achieved  error  rates  with  a  threshold  of  0.05,  based  on  5,000  iterations  at  each 

integer  level  of  3  to  156  cells. 
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V.  CONCLUSIONS  AND  FUTURE  WORK 


This  chapter  summarizes  the  findings  of  the  computational  study,  as  well  as  offers 
some  possible  areas  for  future  study,  for  both  of  the  models  presented  in  this  thesis. 

A.  CONCLUSIONS 

1.  Single  Markov  Target  Model 

Overall,  we  conclude  that  the  stochastic  approximation  algorithm  provides 
sufficiently  improved  solutions  over  all  other  strategies  for  large  search  budgets.  While 
this  may  not  be  significant  on  the  tactical  level  (which  is  often  characterized  by  a 
constrained  search  budget),  it  is  certainly  appropriate  on  the  operational-strategic  level 
wherein  one  is  concerned  with  long-term  Intelligence,  Surveillance,  and  Reconnaissance 
operations  and  Target  of  Interest  pattern  recognition.  Additionally  noteworthy  is  that,  for 
circumstances  in  which  the  stochastic  approximation  algorithm  is  unavailable,  the  naive 
estimate  provides  sufficiently  improved  solutions  over  the  unifonn  random  strategy  for 
large  search  budgets,  and  better  solutions  than  the  stochastic  approximation  strategy  for 
small  search  budgets. 

2.  Single  Static  Target  Model 

The  sequential  eliminating  procedures  presented  in  this  thesis  provide  efficient 
results  guaranteed  to  meet  desired  error  rate  thresholds  for  a  variety  of  scenarios.  While 
the  gap  between  the  error  threshold  and  the  achieved  error  rates  might  conceivably  be 
tightened,  any  efficiency  gained  in  terms  of  reducing  the  number  of  looks  required  is 
likely  not  operationally  significant  when  compared  with  the  risk  of  violating  the 
threshold.  We  showed  that  a  naive,  non-eliminating  method  not  only  demonstrates 
consistently  higher  error  rates  for  the  same  total  number  of  looks  for  all  appropriate 
values  of  both  number  of  area-cells  and  error  tolerance;  there  is  also  no  guarantee  that 
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the  naive  procedure’s  results  meet  the  operator-prescribed  error  tolerance.  It  is  therefore 
reasonable  to  conclude  that  the  sequential  eliminating  procedure  is  more  efficient  than  the 
naive  allocation  method. 

B.  AREAS  FOR  POSSIBLE  FUTURE  STUDY 

1.  Single  Markov  Target  Model 

As  referred  to  throughout  this  thesis,  there  are  several  areas  deserving  further 
study  regarding  our  Single  Markov  Target  model  that  we  were  simply  unable  to  treat  here 
due  to  time  and  scope  limitations.  These  include,  for  example: 

•  Study  the  effects  of  pre-steady-state  target  Markovian  movements. 

•  Study  the  effects  non-Markovian  movements. 

•  Study  the  effect  of  counter-detection;  say,  a  target  of  interest  becomes  alerted 
to  the  presence  of  the  sensor  with  a  certain  probability  following  a  look  into 
the  area-cell  containing  the  target  of  interest,  and  subsequently  moves 
according  to  a  different  transition  matrix. 

•  Allow  for  spatial  and/or  temporal  correlation  among  sensor  observations. 

•  Consider  noise  within  the  observed  measurements  of  sensor  sensitivity  and 
specificity. 

•  Determine  breakpoint  criteria  for  when  the  adaptive  algorithm  is 
advantageous  (i.e.,  what  constitutes  a  “large”  search  budget). 

•  Consider  multiple  sensors  and/or  multiple  targets. 

2.  Single  Static  Target  Model 

As  with  the  Single  Markov  Target  model,  areas  for  possible  further  study  include 
allowing  for  multiple  targets  and/or  sensors,  and  modeling  sensor  sensitivity  and 
specificity  stochastically.  It  may  be  of  interest  to  model  counter-detection  by,  say, 
decreasing  the  associated  value  of  sensor  sensitivity  with  a  certain  probability  following  a 
look  into  the  area-cell  containing  the  target  of  interest.  Additionally,  a  formal  sensitivity 
analysis  is  recommended  in  order  to  determine  any  operational  significant  efficiency 
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gained  by  tightening  the  slack  between  observed  error  rate  and  the  error  tolerance  in 
order  to  decrease  the  expected  number  of  looks.  Finally,  it  may  be  worthwhile  to 
investigate  other  ways  in  which  to  specify  error  tolerances  for  the  multiple-cell  case  such 
as  allowing  the  error  tolerance  to  vary  by  area-cell. 
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APPENDIX  A 


The  following  is  MATLAB  code  used  for  the  adaptive  algorithm,  using  a  four-cell 
example  problem. 

function  bos=markov ( ) 
iter=7000;%  number  of  iterations 
reps=7000 

ss=[0.1  0.4  0.3  0.2];%target  steady  state  distn  (unknown  to  searcher) 
a=[0.8  0.94  0.7  0 . 95 ] ; %sensitivity 
b=[0.1  0.04  0.15  0 . 05] ; %1-specif icity 

prior  =  [.4,  .1,  .2,  ,3];%target  pmf,  (known  to  searcher) 

mu=a. * (ss) +b . * (1-ss) ; %true  mu,  the  value  to  which  #  of  detects  will 

converge 

dim=size (ss,  2)  ;  %  #  area-cells 
epsilon=0 . 05 ;  %absolute  error  tolerance 

lb=b+(a-b)  . *  ( ss-epsilon)  %  lower  bound  of  adequate  allocation 

ub=b+ (a-b)  . *  (ss+epsilon) %  upper  bound  of  adequate  allocation 

iub=ub . *log (ub . /mu) + ( 1-ub) . *log ( ( 1-ub) . / ( 1-mu) ) ; %ld  rate  function  of  ub 

ilb=lb. *log (lb. /mu) + (1-lb) . *log ( (1-lb) ./ (1-mu) ); %ld  rate  function  of  lb 

truedist=min ( ilb, iub) ;  %  true  Id  rate  function 

truef requencies= (1 . /truedist) . /sum ( 1 . /truedist) ; %true  optimal  search 
frequencies 

nf reqs= ( 1 . / (a-b) ) . /sum ( 1 . / (a-b) ) ; %naive  search  frequencies 
for  k=l : reps 

x=b+(a-b) . *prior;%  initializes  x  for  adaptive  based  on  prior  pmf 
xr=x;%  initializes  x  for  random 
xt=x;%  initializes  x  for  a  priori 
xn=x; 

gub=x+ (a-b) *epsilon; 
glb=x- (a-b) *epsilon; 

ixub=gub. *log (gub. / x) + (1-gub) . *log ( (1-gub) . / (1-x) ) ; %ld  rate 
function  of  ub 

ixlb=glb . *log (gib . /x) + ( 1-glb) . *log ( ( 1-glb) . / (1-x) ) ; %ld  rate 
function  of  ub 

dist=min ( ixub, ixlb) ;  %ld  rate  function  of  min 

bucket=dist . A ( -1 ) /sum (dist . A ( -1 ) ) ; %pmf  for  Id  rate  function 

p5=[0,  truef requencies ( 1 : dim-1 )] ;  %interval  setup  for  a  priori 

p6=truef requencies ( 1 : dim) ;  %interval  setup  for  a  priori 

p7=[0,  nfreqs ( 1 : dim-1 )] ;  %interval  setup  for  naive 

p8=nf reqs ( 1 : dim) ;  %interval  setup  for  a  naive 

s=ones ( 1 , dim) ;  %init  sample  sizes  to  one  for  each  cell 
sr=s;%init  sample  sizes  to  one  for  each  cell 
st=s;%init  sample  sizes  to  one  for  each  cell 
sn=s;  %init  sample  sizes  to  one  for  each  cell 


for  i=l : iter 

pi— [ 0 ,  bucket ( 1 : dim-1 )]; %interval  setup  for  adaptive 
p2=bucket ( 1 : dim) ; %interval  setup  for  adaptive 
u=rand;%rand  for  xi 
u3=rand; %rand  for  a  priori  index 
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index=sum ( ( 1 : dim) 
indexr=unidrnd ( 4 ) 


* (cumsum (pi ) <=u) . * (u<cumsum (p2 ) ) ) ; %xi 
%randint ( 1 , 1 ,  [ 1 , 4 ] ) ; %random  uniform  cell 


,  * (u3<cumsum (p6) ) ) ; %cell 
, * (u3<cumsum (p8 ) ) ) ; %cell 


indext=sum ( ( 1 : dim) . * (cumsum (p5 ) <=u3 ) 
based  on  a  priori  optimal  allocation 

indexn=sum ( ( 1 : dim) . * (cumsum (p7 ) <=u3 ) 
based  on  naive  est 

s ( index) =s ( index) +1 ; %update  sample  size  bernoulli  for  adaptive 
sr ( indexr ) =sr ( indexr ) +1 ; %update  sample  size  for  random 
st (indext) =st (indext) +1; %update  sample  size  for  a  priori 
sn ( indexn) =st ( indexn) +1 ; %update  sample  size  for  naive 


r=rand<=mu ( index) ; %detection  bernoulli  for  adaptive 
rr=rand<=mu ( indexr ); %detection  bernoulli  for  random 
rt=rand<=mu (indext) ; 
rn=rand<=mu (indexn) ; 

x (index)  =  x ( index) +( r-x ( index) ) /s ( index) ; %update  avg  for 

adaptive 

xr (indexr)  =  xr (indexr) + (rr-xr (indexr) ) /sr (indexr) ; %update  avg 
for  random 

xt (indext)  =  xt (indext) + (rt-xt (indext) ) /st (indext) ; %update  avg 
for  a  priori 

xn (indexn)  =  xt (indexn) + (rn-xn (indexn) ) /sn (indexn) ; %update  avg 
for  a  priori 

gub (index) =x (index) + (a (index) -b (index) ) *epsilon; 
gib (index) =x (index) - (a (index) -b (index) ) *epsilon; 

ixlb (index) =glb (index) . *log (gib (index) . /x (index) )  +  (1- 
glb (index) ). *log ( (1-glb (index) )./ (1-x (index) )); %ld  rate  function  of  lb 
ixub (index) =gub (index) . *log (gub (index) . /x (index) ) + ( 1 — 
gub (index) ). *log ( (1-gub (index) )./ (1-x (index) )); %ld  rate  function  of  ub 

dist=min ( ixub, ixlb) ; %1 . d .  rate  function  update 

bucket=dist . A ( -1 ) /sum (dist . A ( -1 ) ) ; %pmf  for  Id  rate  function 

pihat= (x-b) . / (a-b) ; 

pihatr= (xr-b) . / (a-b) ; 

pihatt= (xt-b) . / (a-b) ; 

pihatn= (xn-b) . / (a-b) ; 

pihats (i, : ) =pihat; 

pihatsr (i, : ) =pihatr; 

pihatst (i, : ) =pihatt; 

pihatsn (i, : ) =pihatn; 

wrong (k, i) =mean (abs (pihat-ss) >epsilon) ; 
wrongr (k, i) =mean (abs (pihatr-ss) >epsilon) ; 
wrongtl (k, i) =mean (abs (pihatt-ss) >epsilon)  ; 
wrongn (k, i) =mean (abs (pihatn-ss) >epsilon)  ; 
wrongt (k, i) = (exp (-i . / sum (1 . /truedist) ) ) ; 

end 


end 
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pwrong=mean (wrong)  ; 

pwrongr=mean (wrongr)  ; 

pwrongt=mean (wrongt)  ; 

pwrongtl=mean (wrongtl )  ; 

pwrongn=mean (wrongn)  ; 

pihat 

%pihatr 

%pihatt 

ss 

true frequencies 
frequencies= (s-1) /iter 
truemu=ones (iter, 1) *mu; 
sstate=ones (iter, 1) *ss; 

truef reqs=ones (iter, 1) *truef requencies ; 

O, 

o 

plot ( 1 : iter ,  log (pwrong)  ,  ' b ' , 1 : iter , log (pwrongr) ,  ' r ' , 1 : iter , log (pwrongt) 
, ' g ' , 1 : iter , log ( pwrongt 1 ), 'c',l:iter,-(l:iter)/ sum ( 1 . / truedist ) , ' y ' ) 

%  plot(l:iter,log (pwrong) , ' b ' , 1 : iter , log (pwrongr) , 'r',l:iter,- 
( 1 : iter ) / sum ( 1 . / truedist)  ,  ' g '  ) 

%  count 

%  countr 

%  countt 

%  plot (perrort) 

%  plot (sstate) 

%  avgtgt=tgt . /iter 

end 
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APPENDIX  B 


The  following  is  MATLAB  code  used  for  the  sequential  eliminating  procedure, 
varying  the  number  of  area-cells  from  three  to  156,  with  the  TOI  located  in  ACr 

function  bos=seq3() 

reps=5000;%  number  of  replications 


at=[0. 764377 

0.861500282 

0.802665581 

0.921926775 

0.92494074 

0.771560887 

0.833174881 

0.895457497 

0.876803421 

0.962819268 

0.837080099 

0.828107202 

0.868960987 

0.903890473 

0.817516651 

0.771303387 

0.880859016 

0.965729992 

0.79544627 

0.762326265 

0.883559472 

0.864397497 

0.942816637 

0.884480939 

0.907405577 

0.869440723 

0.785080974] 


369  0.78013237 
0.7940199  0. 
0.932069314  0. 
0.858781837  0. 
0.827953419  0. 
0.902829194  0. 
0.789552179  0. 
0.818231974  0. 
0.885381608  0. 
0.930193065  0. 
0.953565399  0. 
0.799289558  0. 
0.863498849  0. 
0.812371407  0. 
0.771200914  0. 
0.829035747  0. 
0.768251293  0. 
0.887229029  0. 
0.978719313  0. 
0.809792289  0. 
0.935584267  0. 
0.914056384  0. 
0.831512882  0. 
0.967052613  0. 
0.86379577  0. 
0.778185422  0. 
; %sensitivity 


0.763048438  0.829653856  0.783187 


778956788  0. 
857595049  0. 
857607031  0. 
872731785  0. 
883563397  0. 
945467317  0. 
827209992  0. 
987431778  0. 
852155027  0. 
752180624  0. 
857286381  0. 
874244484  0. 
750526499  0. 
781980008  0. 
834196225  0. 
80372802  0. 
855403358  0. 
915907164  0. 
914770558  0. 
847992869  0. 
868620928  0. 
942243958  0. 
943980301  0. 
759258625  0. 
800988437  0. 


980568175 

861134275 

894258574 

767612636 

879007539 

946057278 

913855436 

953051 

83147321 

755233645 

82134985 

796966453 

932096997 

765662689 

833510047 

760343658 

821667225 

791414375 

861269546 

773887555 

943567157 

940664075 

886883281 

908502666 

885927148 


0.767952462 

0.901862303 

0.760550857 

0.851690738 

0.906012706 

0.930473901 

0.826787338 

0.906448316 

0.951727379 

0.755896192 

0.929926846 

0.983966875 

0.848402882 

0.845692943 

0.918708516 

0.945212372 

0.940151288 

0.768399415 

0.754924019 

0.935168021 

0.792172003 

0.965565544 

0.768455672 

0.90579148 

0.811211476 


936 

0.882596083 

0.832543914 

0.77867332 

0.809917345 

0.9615216 

0.945149796 

0.952930998 

0.770497982 

0.923018754 

0.889144868 

0.929295357 

0.76516679 

0.812296006 

0.965458891 

0.891839986 

0.840359708 

0.824287152 

0.783205648 

0.848123228 

0.987139592 

0.826948986 

0.973488383 

0.92670777 

0.817439813 

0.802932309 


bt=[0. 906654 

0.913719876 

0.937396994 

0.906367741 

0.934650746 

0.907941657 

0.955030441 

0.9732597 

0.970323846 

0.901042224 

0.978190054 

0.971473421 

0.935198232 

0.979286096 

0.914650274 


853  0.9852728 
0.971642453  0 
0.896209577  0 
0.891722103  0 
0.906862242  0 
0.919534438  0 
0.89127477  0 
0.965857206  0 
0.962837446  0 
0.944637104  0 
0.968984315  0 
0.929638624  0 
0.95602367  0 
0.92988011  0 
0.896422526  0 


9  0.91816009 
.908758043  0. 
.929611542  0. 
.90094522  0. 
.917578657  0. 
.964367434  0. 
.909064212  0. 
.892254905  0. 
.941569896  0. 
.94282935  0. 
.919398871  0. 
.981566707  0. 
.918609615  0. 
.894958239  0. 
.937486111  0. 


0.902462 

934030489 

985859583 

93156408 

981811892 

986505691 

905190465 

950801841 

935252889 

979415014 

978097185 

902712755 

910620992 

90850487 

932364963 


055  0.91547 

0.90977892 

0.909468618 

0.905770115 

0.960555418 

0.904215257 

0.899806225 

0.987436101 

0.942213234 

0.939440625 

0.98411116 

0.902254802 

0.983065473 

0.896762726 

0.941406866 


7538 

0.934459462 

0.93694572 

0.972761166 

0.975909378 

0.899192437 

0.964836142 

0.914702487 

0.975326054 

0.984336508 

0.988082341 

0.926404132 

0.905257303 

0.974151238 

0.939012372 
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0.91097013 

0.973041428 

0.918057093 

0.961941928 

0.89766265 

0.931611186 

0.988183384 

0.918635311 

0.978966905 

0.984093059 

0.92861628 

0.985435625] 


0.981140789 

0.958736224 

0.891750316 

0.92095397 

0.917726817 

0.899762023 

0.988132645 

0.987031639 

0.897238205 

0.895391585 

0.964018148 


0.898929362 
0.925715353 
0.932049333 
0.938774655 
0.964171433 
0.935003204 
0.898435955 
0.909438164 
0.979359218 
0.898494613 
945909754 


0.900527968 

0.966138684 

0.986324799 

0.918541468 

0.985582852 

0.943984386 

0.893955496 

0.976258074 

0.921552236 

0.957411131 

0.934066873 


0.949253233 

0.987483359 

0.982544329 

0.989795575 

0.903957667 

0.959889236 

0.916125454 

0.907865355 

0.984621555 

0.988563306 

0.933560749 


0.914888938 

0.958895003 

0.937989753 

0.941646451 

0.902163265 

0.938170377 

0.909072979 

0.925333803 

0.978638513 

0.98491185 

0.893793761 


0 

1-specificity 


dim=size (at, 2 ) ; 

alpha=0 . 05 ; 

capB= (dim-1 ) / alpha-1 ; 


target=zeros ( 1 , dim) ; %1  if  target  present 
target ( 1 ) =1 ; %1  if  target  present 
mu=at . * (target) +bt . * (1-target) ; 

o, _ 

o 

for  z=3:dim 

oa=0;%keep  track  of  errors  for  error  rate  calc 
capB= ( z-1 ) / alpha- 1 ; 
z 

a=at  ( 1 :  z )  ; 
b=bt  ( 1 : z )  ; 
for  k=l : reps 


c=ones ( 1 , z ) ; %indicators ;  "1"  means  that  cell  is  still  a 
contender 

s=zeros (1, z) ;%number  of  detections 
looks=0; %#looks 
stage=0 ; 


while  (sum(c) >1) 

stage=stage+l ; 

%  c 

for  i=l : z 

if  c(i)==l 

looks=looks+l; 
r ( i ) = ( rand<=mu ( i ) ) ; 
s  (i)  =s  (i)  +r  (i)  ; 

like (i)=(a(i) As (i) * (l-a(i) ) A (stage- 
s (i) ) ) / (b(i) As (i) * (l-b(i) ) A ( stage- s (i) ) ) ; %likelihood  ratios 
end 

end 

maxrat=find (like==max (like) )  ; 
for  q  1 :  z. 

if  c (q) ==1 

ratio (q) =like (maxrat (1) ) . /like (q) ; %odds  ratios 

end 

end 

for  j  =1 : z 
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losers,"  but 


if  ratio ( j ) >=capB  &&  sum (c) >1 ; %removes 
keeps  from  removing  the  last  contender 

c ( j ) =0 ; 

end 

end 


end 

numLooks (k) =looks; 
declareTgt (k) =find (c==l) ; 

oa=oa+ (declareTgt (k) ~=1) ; 


end 

oarate ( z ) =oa . / reps ; 
avgLooks ( z ) =mean (numLooks ) ; 

end 

plot (avgLooks) 
end 
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APPENDIX  C 


The  following  tables  describe  the  non-zero  columns  of  the  transition  matrix  used 
in  the  adaptive  problem  of  this  thesis. 


From\To 

45 

57 

58 

70 

71 

76 

77 

78 

88 

89 

90 

100 

101 

102 

103 

104 

112 

113 

114 

1 

0 

0 

0 

0 

0 

0 

0.1 

0 

0.1 

0 

0.1 

0 

0 

0.1 

0.1 

0 

0.2 

0.2 

0.1 

2 

0 

0.1 

0 

0 

0.1 

0 

0 

0 

0.1 

0 

0.1 

0 

0 

0 

0 

0 

0.1 

0.4 

0.1 

3 

0 

0 

0 

0 

0 

0 

0.1 

0 

0 

0 

0 

0.2 

0.1 

0 

0 

0 

0.2 

0 

0.4 

4 

0 

0 

0 

0 

0 

0.1 

0 

0 

0 

0.1 

0 

0.1 

0.1 

0.1 

0 

0 

0 

0.1 

0.4 

5 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0.1 

0.1 

0.1 

0 

0.1 

0 

0.2 

0.4 

6 

0 

0 

0 

0 

0 

0.1 

0 

0 

0.1 

0.1 

0 

0.1 

0 

0 

0.1 

0.1 

0 

0 

0.4 

7 

0 

0 

0 

0.1 

0 

0 

0.1 

0 

0 

0.1 

0.1 

0.1 

0 

0.1 

0 

0.1 

0 

0 

0.3 

8 

0 

0 

0 

0 

0 

0 

0.1 

0 

0 

0 

0 

0 

0.2 

0 

0.1 

0.1 

0.2 

0 

0.3 

9 

0 

0 

0 

0 

0.1 

0 

0 

0 

0 

0.1 

0.1 

0 

0.1 

0 

0 

0 

0.3 

0 

0.3 

10 

0 

0 

0 

0 

0 

0 

0.1 

0 

0 

0.1 

0.1 

0 

0 

0.1 

0.1 

0 

0 

0.4 

0.1 

11 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0.1 

0.2 

0 

0.1 

0.1 

0 

0.2 

0.1 

0.2 

12 

0 

0 

0 

0.1 

0 

0.1 

0 

0.1 

0 

0 

0 

0.1 

0 

0 

0.1 

0.1 

0.2 

0.1 

0.1 

13 

0 

0 

0 

0.1 

0 

0.1 

0 

0.1 

0 

0.1 

0.1 

0 

0 

0.1 

0.1 

0 

0.1 

0 

0.2 

14 

0 

0 

0 

0 

0 

0 

0 

0 

0.1 

0.1 

0.1 

0.1 

0.1 

0 

0.1 

0 

0 

0.1 

0.3 

15 

0 

0 

0 

0 

0.1 

0 

0 

0 

0.1 

0.1 

0.1 

0 

0 

0 

0.2 

0.1 

0.1 

0.1 

0.1 

16 

0 

0 

0 

0 

0.1 

0.1 

0.1 

0 

0.1 

0.1 

0 

0 

0.1 

0.1 

0 

0 

0 

0 

0.3 

17 

0 

0 

0 

0 

0 

0.1 

0.1 

0 

0 

0 

0.1 

0 

0 

0 

0 

0.3 

0.1 

0.1 

0.2 

18 

0 

0 

0.1 

0 

0 

0 

0.1 

0.1 

0 

0 

0 

0 

0 

0 

0.1 

0 

0.1 

0.3 

0.2 

19 

0 

0 

0 

0 

0 

0 

0.1 

0 

0 

0.1 

0 

0.1 

0.1 

0.1 

0.1 

0.1 

0.1 

0.1 

0.1 

20 

0 

0 

0 

0 

0 

0 

0 

0 

0.1 

0.1 

0.1 

0 

0 

0 

0.1 

0 

0 

0.3 

0.3 

21 

0 

0 

0.1 

0 

0 

0 

0 

0 

0 

0 

0.1 

0 

0 

0.1 

0.2 

0.1 

0.1 

0.1 

0.2 

22 

0 

0 

0.1 

0.1 

0 

0 

0 

0 

0 

0 

0 

0.1 

0 

0.2 

0.1 

0 

0.1 

0.2 

0.1 

23 

0 

0.1 

0 

0 

0 

0 

0 

0 

0.1 

0 

0 

0 

0.1 

0 

0.1 

0.2 

0.1 

0 

0.3 

24 

0 

0 

0 

0 

0 

0 

0 

0 

0.1 

0.1 

0.1 

0 

0 

0 

0.1 

0 

0.1 

0 

0.5 

25 

0 

0 

0 

0 

0.1 

0 

0.1 

0.1 

0 

0 

0 

0.1 

0 

0 

0.1 

0.1 

0.1 

0.1 

0.2 

26 

0 

0 

0 

0 

0.1 

0 

0 

0.1 

0.1 

0.1 

0 

0.1 

0 

0 

0.1 

0.1 

0 

0.2 

0.1 

27 

0 

0 

0 

0 

0 

0 

0.1 

0.1 

0 

0 

0.1 

0.1 

0 

0 

0.2 

0.1 

0 

0.2 

0.1 

28 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0.1 

0 

0.2 

0.2 

0 

0.1 

0 

0.3 

0.1 

29 

0 

0 

0 

0 

0 

0 

0 

0 

0.1 

0 

0.1 

0 

0.1 

0.1 

0.1 

0.2 

0 

0.1 

0.2 

30 

0 

0 

0 

0.1 

0 

0 

0 

0 

0 

0 

0 

0 

0.2 

0.1 

0 

0 

0 

0.5 

0.1 

31 

0 

0 

0 

0 

0.1 

0 

0.1 

0.1 

0 

0 

0.1 

0 

0.1 

0 

0.1 

0.1 

0 

0.1 

0.2 

32 

0 

0 

0 

0 

0 

0 

0.1 

0 

0.1 

0 

0 

0 

0.2 

0.1 

0 

0.2 

0 

0.1 

0.2 

33 

0 

0 

0 

0 

0 

0 

0 

0 

0.1 

0.1 

0.1 

0 

0.1 

0.1 

0.1 

0 

0.1 

0 

0.3 

34 

0 

0 

0 

0 

0 

0 

0 

0.1 

0 

0.1 

0 

0 

0 

0.2 

0 

0.2 

0.1 

0.2 

0.1 

35 

0 

0 

0.1 

0.1 

0 

0 

0.1 

0 

0.1 

0 

0 

0 

0 

0 

0.2 

0.1 

0 

0.2 

0.1 

36 

0 

0 

0 

0 

0 

0 

0.1 

0.1 

0.1 

0.1 

0.1 

0 

0.1 

0.1 

0.1 

0 

0 

0.1 

0.1 

37 

0 

0.1 

0 

0 

0 

0 

0 

0 

0.1 

0.1 

0 

0 

0.1 

0.1 

0 

0.2 

0.1 

0 

0.2 

38 

0 

0 

0 

0.1 

0.1 

0 

0 

0 

0 

0.1 

0 

0.1 

0 

0.1 

0.1 

0.1 

0.1 

0.1 

0.1 

39 

0 

0 

0 

0 

0 

0 

0 

0 

0.1 

0 

0.1 

0 

0.2 

0 

0 

0 

0 

0.3 

0.3 

40 

0 

0 

0 

0 

0.1 

0 

0 

0 

0 

0 

0.1 

0 

0.2 

0 

0.1 

0.2 

0 

0.1 

0.2 

41 

0 

0 

0 

0 

0 

0 

0.1 

0 

0 

0 

0 

0.1 

0 

0 

0 

0.3 

0.2 

0 

0.3 

42 

0 

0.1 

0 

0 

0 

0.1 

0 

0 

0 

0 

0.1 

0.1 

0.1 

0 

0.1 

0.1 

0 

0 

0.3 

43 

0 

0 

0 

0.1 

0 

0.1 

0.1 

0 

0.1 

0 

0 

0 

0 

0 

0.2 

0.1 

0.1 

0.1 

0.1 

44 

0.1 

0 

0 

0 

0 

0.1 

0 

0.1 

0 

0 

0.1 

0 

0.1 

0.1 

0 

0 

0.2 

0 

0.2 

45 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0.1 

0 

0 

0.2 

0.1 

0.1 

0.1 

0.2 

0.1 

0.1 

46 

0.1 

0 

0 

0 

0 

0 

0 

0.1 

0 

0.1 

0 

0.1 

0 

0.1 

0 

0 

0.1 

0.2 

0.2 

47 

0 

0 

0 

0 

0 

0 

0 

0.1 

0 

0 

0 

0 

0.2 

0.1 

0 

0.1 

0.1 

0.1 

0.3 

48 

0 

0 

0 

0 

0 

0 

0.1 

0.1 

0 

0 

0.1 

0.1 

0 

0 

0 

0 

0.3 

0 

0.3 

49 

0 

0 

0 

0 

0 

0 

0 

0.1 

0.1 

0 

0 

0.1 

0.1 

0.1 

0 

0.1 

0 

0 

0.4 

50 

0 

0 

0 

0 

0 

0.1 

0 

0 

0 

0.1 

0.1 

0.1 

0 

0.1 

0 

0.2 

0 

0.2 

0.1 

51 

0 

0 

0 

0 

0 

0 

0 

0.1 

0 

0 

0 

0.1 

0.1 

0 

0.1 

0.2 

0 

0.2 

0.2 

52 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0.2 

0.1 

0.1 

0 

0.1 

0.2 

0 

0.2 

0.1 

Table  5.  TOI  transition  matrix  (non-zero  columns  only).  (Sheet  1  of  3). 
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From\To 

45 

57 

58 

70 

71 

76 

77 

78 

88 

89 

90 

100 

101 

102 

103 

104 

112 

113 

114 

53 

0 

0 

0 

0 

0.1 

0 

0 

0.1 

0.1 

0.1 

0 

0.1 

0 

0.1 

0 

0 

0 

0.3 

0.1 

54 

0 

0.1 

0 

0 

0 

0 

0 

0.1 

0 

0 

0 

0 

0 

0.1 

0.1 

0.1 

0.1 

0 

0.4 

55 

0 

0 

0 

0 

0 

0 

0 

0.1 

0.1 

0.1 

0.1 

0 

0 

0 

0 

0 

0.1 

0 

0.5 

56 

0 

0 

0 

0 

0.1 

0 

0 

0 

0.1 

0 

0 

0.1 

0 

0.1 

0.2 

0.1 

0 

0 

0.3 

57 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0.1 

0.1 

0 

0.1 

0 

0 

0 

0 

0.2 

0.5 

58 

0 

0 

0 

0 

0.1 

0 

0.1 

0 

0 

0.1 

0.1 

0 

0 

0.1 

0.1 

0.1 

0.1 

0 

0.2 

59 

0 

0 

0 

0 

0 

0 

0 

0 

0.1 

0 

0 

0.1 

0 

0 

0 

0.3 

0.2 

0.1 

0.2 

60 

0 

0 

0 

0.1 

0 

0 

0 

0.1 

0 

0 

0.1 

0.1 

0 

0.1 

0.1 

0.1 

0.1 

0 

0.2 

61 

0 

0 

0 

0 

0 

0.1 

0 

0.1 

0 

0.1 

0.1 

0 

0 

0.1 

0 

0 

0.2 

0 

0.3 

62 

0 

0.1 

0 

0 

0 

0 

0.1 

0 

0 

0.1 

0 

0.1 

0 

0 

0.1 

0 

0.3 

0 

0.2 

63 

0 

0 

0 

0 

0 

0.1 

0.1 

0 

0 

0.1 

0 

0 

0.1 

0 

0.2 

0 

0.1 

0 

0.3 

64 

0 

0 

0.1 

0.1 

0 

0 

0 

0 

0.1 

0 

0 

0.1 

0.1 

0.1 

0 

0 

0 

0 

0.4 

65 

0 

0 

0 

0.1 

0 

0.1 

0.1 

0 

0.1 

0 

0 

0 

0.1 

0.1 

0.1 

0.1 

0 

0 

0.2 

66 

0 

0 

0 

0.1 

0 

0 

0.1 

0 

0.1 

0 

0.1 

0 

0 

0.1 

0 

0.1 

0.2 

0 

0.2 

67 

0 

0 

0.1 

0 

0 

0 

0 

0 

0.1 

0.1 

0 

0 

0.1 

0.1 

0 

0.1 

0 

0.1 

0.3 

68 

0 

0 

0 

0 

0.1 

0 

0.1 

0 

0.1 

0 

0.1 

0.1 

0 

0.1 

0.1 

0 

0.1 

0 

0.2 

69 

0 

0 

0 

0 

0.1 

0 

0 

0 

0 

0.1 

0 

0.1 

0 

0 

0.1 

0.1 

0 

0.4 

0.1 

70 

0 

0 

0 

0 

0.1 

0 

0 

0 

0 

0.1 

0.1 

0 

0.1 

0.1 

0.1 

0 

0 

0.2 

0.2 

71 

0 

0 

0 

0.1 

0 

0 

0.1 

0 

0 

0 

0 

0.1 

0 

0 

0 

0.2 

0.1 

0.2 

0.2 

72 

0 

0 

0 

0 

0.1 

0 

0 

0 

0 

0.1 

0 

0.1 

0 

0.1 

0.1 

0.2 

0 

0.1 

0.2 

73 

0 

0 

0 

0.1 

0 

0.1 

0 

0.1 

0 

0 

0 

0 

0 

0 

0.2 

0.2 

0.1 

0.1 

0.1 

74 

0 

0 

0 

0 

0.1 

0 

0 

0 

0 

0 

0.1 

0.1 

0.1 

0.1 

0.1 

0.1 

0 

0.2 

0.1 

75 

0 

0 

0 

0 

0 

0 

0 

0.1 

0.1 

0.1 

0.1 

0 

0.1 

0 

0.1 

0 

0 

0.2 

0.2 

76 

0 

0 

0 

0 

0 

0 

0 

0.1 

0 

0 

0 

0.1 

0.1 

0 

0 

0.3 

0.2 

0.1 

0.1 

77 

0 

0 

0 

0 

0 

0 

0.1 

0 

0 

0 

0 

0 

0.2 

0.1 

0 

0 

0.1 

0.3 

0.2 

78 

0 

0 

0 

0 

0.1 

0 

0 

0 

0 

0.1 

0 

0.1 

0.1 

0 

0.2 

0 

0 

0 

0.4 

79 

0 

0 

0 

0 

0.1 

0 

0.1 

0 

0.1 

0 

0 

0.1 

0 

0.1 

0.1 

0 

0 

0.2 

0.2 

80 

0 

0 

0 

0.1 

0 

0.1 

0 

0.1 

0 

0.1 

0 

0.1 

0 

0.1 

0 

0 

0.2 

0.1 

0.1 

81 

0 

0 

0 

0 

0 

0 

0 

0.1 

0.1 

0 

0.1 

0 

0 

0 

0.1 

0.1 

0 

0.1 

0.4 

82 

0 

0 

0 

0.1 

0 

0 

0 

0.1 

0 

0.1 

0 

0.1 

0 

0.1 

0.1 

0 

0 

0.2 

0.2 

83 

0 

0 

0 

0 

0 

0 

0.1 

0.1 

0 

0 

0 

0 

0.1 

0.2 

0.1 

0.1 

0 

0 

0.3 

84 

0 

0.1 

0 

0 

0 

0.1 

0 

0 

0.1 

0 

0 

0 

0.1 

0 

0.1 

0 

0.1 

0.2 

0.2 

85 

0 

0 

0 

0.1 

0.1 

0 

0.1 

0 

0 

0 

0.1 

0 

0 

0.1 

0 

0.2 

0 

0 

0.3 

86 

0 

0 

0 

0 

0 

0.1 

0 

0.1 

0.1 

0 

0 

0.1 

0 

0.1 

0 

0 

0 

0.1 

0.4 

87 

0 

0 

0 

0 

0 

0 

0 

0.1 

0 

0 

0 

0.1 

0.1 

0.2 

0.1 

0 

0.2 

0 

0.2 

88 

0 

0 

0.1 

0 

0 

0 

0.1 

0 

0 

0 

0.1 

0 

0 

0 

0 

0.2 

0.1 

0.1 

0.3 

89 

0 

0 

0 

0 

0 

0 

0 

0.1 

0 

0 

0.1 

0 

0.1 

0.1 

0.2 

0.1 

0.1 

0.1 

0.1 

90 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0.1 

0.1 

0.1 

0 

0.3 

0.2 

0 

0.2 

91 

0 

0 

0 

0 

0 

0 

0 

0.1 

0 

0 

0.1 

0 

0.1 

0 

0 

0.1 

0.3 

0 

0.3 

92 

0 

0 

0.1 

0 

0 

0.1 

0 

0 

0.1 

0 

0 

0 

0 

0.1 

0.1 

0 

0 

0.4 

0.1 

93 

0 

0 

0 

0 

0 

0.1 

0 

0 

0 

0.1 

0 

0 

0 

0.2 

0.1 

0.1 

0.1 

0.1 

0.2 

94 

0 

0 

0 

0 

0 

0 

0 

0 

0.1 

0 

0 

0 

0 

0 

0.1 

0.1 

0.4 

0.1 

0.2 

95 

0 

0 

0.1 

0 

0 

0 

0 

0 

0 

0 

0.1 

0 

0.1 

0.1 

0.2 

0 

0 

0.1 

0.3 

96 

0 

0 

0 

0 

0 

0.1 

0 

0 

0 

0 

0 

0.1 

0 

0.2 

0.2 

0.1 

0 

0.2 

0.1 

97 

0 

0 

0.1 

0 

0 

0 

0 

0 

0 

0.1 

0.1 

0.1 

0.1 

0 

0 

0.1 

0.1 

0 

0.3 

98 

0 

0 

0 

0 

0 

0 

0.1 

0 

0.1 

0 

0.1 

0.1 

0.1 

0 

0.1 

0 

0.2 

0.1 

0.1 

99 

0 

0 

0 

0.1 

0.1 

0 

0 

0.1 

0 

0 

0 

0 

0.1 

0 

0.1 

0.2 

0 

0 

0.3 

100 

0 

0 

0 

0 

0 

0.1 

0 

0 

0 

0 

0.1 

0.1 

0.1 

0 

0.1 

0 

0.3 

0 

0.2 

101 

0 

0 

0 

0 

0.1 

0 

0.1 

0 

0 

0 

0.1 

0 

0.1 

0 

0 

0 

0.1 

0.4 

0.1 

102 

0 

0 

0 

0 

0 

0.1 

0 

0 

0.1 

0.1 

0 

0 

0 

0 

0.2 

0.1 

0.1 

0.1 

0.2 

103 

0 

0 

0 

0 

0 

0 

0 

0 

0.1 

0 

0 

0 

0.1 

0.2 

0.1 

0.1 

0 

0 

0.4 

104 

0 

0 

0 

0.1 

0 

0 

0.1 

0.1 

0 

0 

0.1 

0 

0 

0 

0.1 

0.1 

0.1 

0.2 

0.1 

Table  6.  TOI  transition  matrix  (non-zero  columns  only).  (Sheet  2  of  3). 
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From\To 

45 

57 

58 

70 

71 

76 

77 

78 

88 

89 

90 

100 

101 

102 

103 

104 

112 

113 

114 

105 

0 

0.1 

0 

0 

0 

0 

0 

0 

0 

0 

0.1 

0 

0.1 

0.2 

0.1 

0.1 

0.1 

0 

0.2 

106 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0.1 

0.1 

0.1 

0.1 

0 

0.1 

0 

0 

0.1 

0.4 

107 

0 

0 

0 

0 

0.1 

0 

0 

0 

0 

0.1 

0.1 

0.1 

0.1 

0 

0.1 

0.1 

0.1 

0.1 

0.1 

108 

0 

0 

0 

0 

0 

0 

0 

0.1 

0 

0.1 

0 

0.1 

0 

0.1 

0.1 

0 

0.3 

0 

0.2 

109 

0 

0 

0 

0 

0 

0 

0 

0 

0.1 

0.1 

0 

0.1 

0.1 

0 

0 

0.1 

0.2 

0 

0.3 

110 

0 

0.1 

0 

0 

0 

0 

0 

0 

0 

0 

0.1 

0.1 

0.1 

0.1 

0 

0 

0.2 

0 

0.3 

111 

0 

0 

0 

0 

0 

0 

0.1 

0.1 

0 

0 

0.1 

0.1 

0 

0.1 

0.1 

0 

0 

0.3 

0.1 

112 

0 

0 

0 

0 

0.1 

0 

0.1 

0.1 

0 

0 

0 

0.1 

0 

0.1 

0 

0.1 

0.1 

0.1 

0.2 

113 

0 

0 

0 

0.1 

0 

0 

0 

0.1 

0 

0.1 

0 

0.1 

0.1 

0 

0.1 

0 

0.1 

0.2 

0.1 

114 

0 

0 

0.1 

0 

0 

0.1 

0 

0.1 

0 

0.1 

0 

0 

0 

0.1 

0.1 

0.1 

0.1 

0 

0.2 

115 

0.1 

0 

0 

0 

0.1 

0 

0 

0 

0 

0 

0 

0 

0.2 

0 

0.1 

0.2 

0.1 

0 

0.2 

116 

0 

0 

0 

0 

0.1 

0.1 

0 

0.1 

0.1 

0.1 

0 

0 

0 

0 

0 

0 

0 

0.1 

0.4 

117 

0 

0 

0 

0 

0.1 

0.1 

0 

0 

0.1 

0 

0 

0 

0.1 

0 

0.1 

0.1 

0.1 

0.2 

0.1 

118 

0 

0 

0 

0 

0 

0.1 

0 

0.1 

0.1 

0 

0.1 

0 

0 

0.1 

0.1 

0.1 

0 

0.1 

0.2 

119 

0 

0 

0 

0 

0 

0 

0.1 

0 

0 

0.1 

0.1 

0 

0 

0.1 

0 

0.2 

0.1 

0 

0.3 

120 

0 

0 

0 

0 

0 

0 

0.1 

0 

0 

0.1 

0 

0 

0 

0.2 

0 

0.1 

0 

0 

0.5 

121 

0 

0.1 

0 

0 

0 

0 

0 

0 

0 

0.1 

0 

0.1 

0.1 

0 

0.1 

0 

0.2 

0.2 

0.1 

122 

0 

0 

0 

0.1 

0 

0.1 

0 

0 

0.1 

0.1 

0.1 

0 

0.1 

0 

0 

0.1 

0 

0 

0.3 

123 

0 

0 

0 

0.1 

0 

0 

0 

0 

0 

0 

0.1 

0 

0.1 

0 

0.1 

0.1 

0 

0.3 

0.2 

124 

0 

0 

0 

0 

0.1 

0 

0 

0 

0.1 

0 

0.1 

0.1 

0.1 

0 

0 

0 

0 

0 

0.5 

125 

0 

0 

0 

0 

0 

0 

0 

0.1 

0 

0.1 

0.1 

0 

0.1 

0.1 

0.1 

0.1 

0.1 

0.1 

0.1 

126 

0 

0 

0.1 

0 

0.1 

0.1 

0 

0 

0 

0 

0.1 

0 

0.1 

0 

0 

0 

0.1 

0.3 

0.1 

127 

0 

0 

0 

0 

0 

0.1 

0.1 

0 

0.1 

0 

0.1 

0 

0 

0.1 

0 

0 

0 

0.1 

0.4 

128 

0 

0 

0.1 

0 

0 

0 

0.1 

0 

0.1 

0 

0 

0 

0.1 

0.1 

0.1 

0.1 

0.1 

0.1 

0.1 

129 

0 

0 

0 

0 

0.1 

0.1 

0.1 

0 

0 

0 

0.1 

0 

0.1 

0.1 

0 

0 

0.1 

0.2 

0.1 

130 

0 

0.1 

0 

0 

0 

0.1 

0 

0 

0.1 

0 

0 

0 

0.1 

0 

0.2 

0 

0.1 

0.1 

0.2 

131 

0 

0 

0 

0 

0.1 

0 

0 

0.1 

0 

0.1 

0 

0.1 

0.1 

0.1 

0.1 

0.1 

0 

0.1 

0.1 

132 

0 

0 

0 

0 

0 

0.1 

0.1 

0.1 

0 

0 

0 

0 

0 

0.2 

0 

0.1 

0.1 

0 

0.3 

133 

0 

0 

0 

0 

0 

0.1 

0 

0 

0 

0 

0.1 

0 

0 

0.1 

0.1 

0 

0.3 

0.1 

0.2 

134 

0 

0 

0 

0 

0 

0 

0 

0.1 

0 

0 

0 

0 

0.1 

0 

0 

0.2 

0.2 

0.3 

0.1 

135 

0 

0.1 

0 

0.1 

0 

0 

0 

0 

0 

0.1 

0.1 

0 

0 

0.1 

0 

0 

0.1 

0.2 

0.2 

136 

0 

0 

0 

0 

0 

0 

0.1 

0 

0.1 

0 

0 

0.1 

0.1 

0 

0 

0.2 

0.2 

0.1 

0.1 

137 

0 

0 

0 

0 

0 

0.1 

0 

0 

0.1 

0 

0.1 

0.1 

0 

0 

0 

0.2 

0.1 

0.2 

0.1 

138 

0 

0 

0 

0.1 

0 

0 

0 

0 

0 

0 

0.1 

0.1 

0 

0.2 

0.1 

0 

0.1 

0 

0.3 

139 

0 

0 

0.1 

0 

0 

0.1 

0 

0.1 

0 

0.1 

0 

0 

0 

0.1 

0.1 

0.1 

0 

0.2 

0.1 

140 

0 

0 

0.1 

0 

0 

0 

0 

0 

0.1 

0.1 

0 

0 

0 

0.1 

0.2 

0.1 

0 

0.2 

0.1 

141 

0 

0.1 

0 

0 

0 

0.1 

0 

0 

0.1 

0 

0 

0 

0.1 

0 

0 

0.2 

0.2 

0 

0.2 

142 

0 

0 

0 

0 

0 

0 

0 

0 

0.1 

0.1 

0.1 

0.1 

0 

0 

0.1 

0.1 

0 

0.3 

0.1 

143 

0 

0 

0 

0 

0 

0 

0 

0.1 

0 

0.1 

0 

0.1 

0 

0.2 

0.1 

0 

0.1 

0 

0.3 

144 

0 

0.1 

0 

0 

0.1 

0 

0 

0 

0 

0 

0 

0 

0.1 

0 

0.1 

0 

0.2 

0.1 

0.3 

145 

0 

0 

0 

0 

0 

0.1 

0.1 

0 

0 

0.1 

0 

0.1 

0.1 

0.1 

0 

0 

0 

0.2 

0.2 

146 

0 

0 

0.1 

0 

0 

0 

0 

0 

0 

0 

0 

0.1 

0.1 

0 

0.2 

0.1 

0 

0 

0.4 

147 

0 

0 

0 

0 

0.1 

0 

0 

0.1 

0 

0 

0 

0 

0.1 

0.1 

0 

0 

0.3 

0.2 

0.1 

148 

0 

0 

0 

0 

0 

0 

0.1 

0 

0.1 

0.1 

0 

0 

0.1 

0.1 

0 

0 

0.2 

0 

0.3 

149 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0.1 

0.1 

0.1 

0 

0.2 

0 

0.2 

0.1 

0.1 

0.1 

150 

0 

0 

0.1 

0 

0 

0 

0 

0.1 

0 

0 

0.1 

0 

0 

0 

0 

0.1 

0.1 

0.1 

0.4 

151 

0 

0.1 

0 

0 

0 

0 

0 

0.1 

0 

0.1 

0 

0 

0 

0.1 

0 

0 

0 

0.4 

0.2 

152 

0 

0 

0 

0.1 

0 

0.1 

0 

0 

0 

0 

0.1 

0 

0.1 

0 

0.1 

0.1 

0 

0.2 

0.2 

153 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0.1 

0 

0.1 

0.1 

0.3 

0 

0.2 

0.2 

154 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0.1 

0 

0.1 

0.1 

0.1 

0.1 

0.1 

0.2 

0 

0.2 

155 

0 

0 

0 

0.1 

0 

0 

0 

0.1 

0 

0 

0.1 

0.1 

0 

0 

0 

0.2 

0.2 

0.1 

0.1 

156 

0 

0 

0 

0.1 

0 

0 

0 

0 

0.1 

0 

0.1 

0 

0 

0 

0.2 

0 

0.1 

0 

0.4 

Table  7.  TOI  transition  matrix  (non-zero  columns  only).  (Sheet  3  of  3). 
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